2024-03-23T07:41:49.215 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T07:41:49.215 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/cap-flush} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362 -- /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/orig.config.yaml 2024-03-23T07:41:49.258 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T07:41:49.339 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362 branch: wip-yuri11-testing-2024-03-21-0851-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/cap-flush} email: yweinste@redhat.com first_in_suite: false job_id: '7617362' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri11-testing-2024-03-21-0851-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds.a sha1: 82316b04821a6c2c1d458a881920454bbadb8837 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 82316b04821a6c2c1d458a881920454bbadb8837 kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri11-testing-2024-03-21-0851-reef sha1: 82316b04821a6c2c1d458a881920454bbadb8837 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1919 sha1: 82316b04821a6c2c1d458a881920454bbadb8837 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri11-testing-2024-03-21-0851-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 82316b04821a6c2c1d458a881920454bbadb8837 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCvGzdXZnCG7efrJf2Bl6Dzu60db9fLBm1CeDGCfKRjVrXXEK7w2OVUopAP2ZAodOEUU7d+cxO2ZO98vccUPrtE= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjJfB8Bx/I2rR6IbWMR3RU/LMDkgYFJUwA6VV18wAW1BPYbLlpVCjOp/Htuc5OR++0aJno1/ErdH334QNCUea0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_cap_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_13:15:24 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-23T07:41:49.339 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa; will attempt to use it 2024-03-23T07:41:49.339 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks 2024-03-23T07:41:49.340 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T07:41:49.341 INFO:teuthology.task.internal:Checking packages... 2024-03-23T07:41:49.364 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '82316b04821a6c2c1d458a881920454bbadb8837' 2024-03-23T07:41:49.365 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T07:41:49.365 INFO:teuthology.packaging:ref: None 2024-03-23T07:41:49.365 INFO:teuthology.packaging:tag: None 2024-03-23T07:41:49.365 INFO:teuthology.packaging:branch: wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T07:41:49.365 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T07:41:49.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T07:41:49.581 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-954.g82316b04 2024-03-23T07:41:49.583 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T07:41:49.589 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T07:41:49.590 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T07:41:49.604 INFO:teuthology.task.internal:Saving configuration 2024-03-23T07:41:49.619 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T07:41:49.625 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T07:41:49.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:36:50.027132', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCvGzdXZnCG7efrJf2Bl6Dzu60db9fLBm1CeDGCfKRjVrXXEK7w2OVUopAP2ZAodOEUU7d+cxO2ZO98vccUPrtE='} 2024-03-23T07:41:49.673 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:36:50.029059', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjJfB8Bx/I2rR6IbWMR3RU/LMDkgYFJUwA6VV18wAW1BPYbLlpVCjOp/Htuc5OR++0aJno1/ErdH334QNCUea0='} 2024-03-23T07:41:49.673 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T07:41:49.679 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-23T07:41:49.679 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-23T07:41:49.679 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T07:41:49.748 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd464a5b160>, signals=[15]) 2024-03-23T07:41:49.748 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T07:41:49.753 INFO:teuthology.task.internal:Opening connections... 2024-03-23T07:41:49.754 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:41:49.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:41:49.843 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:41:49.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:41:49.928 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T07:41:49.933 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-03-23T07:41:49.951 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-03-23T07:41:49.951 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T07:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-03-23T07:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-03-23T07:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T07:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-03-23T07:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T07:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T07:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T07:41:50.008 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-03-23T07:41:50.027 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-03-23T07:41:50.044 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-03-23T07:41:50.044 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-03-23T07:41:50.100 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-03-23T07:41:50.100 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T07:41:50.101 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T07:41:50.102 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-03-23T07:41:50.118 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T07:41:50.127 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T07:41:50.132 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T07:41:50.132 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-03-23T07:41:50.135 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-03-23T07:41:50.156 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T07:41:50.161 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T07:41:50.161 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-03-23T07:41:50.190 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-03-23T07:41:50.306 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T07:41:50.385 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T07:41:50.397 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-23T07:41:50.398 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-03-23T07:41:50.398 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-23T07:41:50.398 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T07:41:50.398 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T07:41:50.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:41:50.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-23T07:41:50.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:41:50.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-23T07:41:50.650 DEBUG:teuthology.task.kernel:sha1 for client.1 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T07:41:50.650 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-23T07:41:50.651 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-03-23T07:41:50.668 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-361.el9.x86_64 2024-03-23T07:41:50.668 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T07:41:50.668 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-23T07:41:50.669 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum upgrade -y linux-firmware 2024-03-23T07:41:50.741 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T07:41:50.741 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-23T07:41:50.742 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-03-23T07:41:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-361.el9.x86_64 2024-03-23T07:41:50.759 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T07:41:50.759 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-23T07:41:50.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum upgrade -y linux-firmware 2024-03-23T07:41:52.987 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-23T07:41:53.004 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-23T07:41:54.430 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-03-23T07:41:54.621 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-23T07:41:55.412 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-23T07:41:55.672 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-23T07:41:57.413 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-23T07:41:57.687 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-23T07:42:06.112 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 6.1 MB 00:04 2024-03-23T07:42:07.605 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-03-23T07:42:09.105 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 762 kB/s | 6.1 MB 00:08 2024-03-23T07:42:10.264 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-03-23T07:42:10.592 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-03-23T07:42:12.299 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-23T07:42:15.239 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-23T07:42:17.207 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-23T07:42:17.418 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 1 Package 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 375 M 2024-03-23T07:42:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-23T07:42:19.394 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 1 Package 2024-03-23T07:42:19.404 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:42:19.405 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 375 M 2024-03-23T07:42:19.405 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-23T07:42:29.166 INFO:teuthology.orchestra.run.smithi105.stdout:linux-firmware-20240219-143.el9.noarch.rpm 38 MB/s | 375 MB 00:09 2024-03-23T07:42:29.167 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-23T07:42:29.167 INFO:teuthology.orchestra.run.smithi105.stdout:Total 38 MB/s | 375 MB 00:09 2024-03-23T07:42:29.168 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-23T07:42:29.252 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-23T07:42:29.252 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-23T07:42:31.625 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-23T07:42:31.626 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-23T07:42:34.122 INFO:teuthology.orchestra.run.smithi132.stdout:linux-firmware-20240219-143.el9.noarch.rpm 22 MB/s | 375 MB 00:16 2024-03-23T07:42:34.123 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-23T07:42:34.124 INFO:teuthology.orchestra.run.smithi132.stdout:Total 22 MB/s | 375 MB 00:16 2024-03-23T07:42:34.124 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-23T07:42:34.206 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-23T07:42:34.207 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-23T07:42:36.658 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-23T07:42:36.658 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-23T07:42:38.002 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-23T07:42:38.243 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-23T07:42:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T07:42:39.569 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T07:42:39.570 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-23T07:42:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T07:42:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:42:39.953 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-03-23T07:42:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-23T07:42:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:42:39.953 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-23T07:42:40.225 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-23T07:42:40.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T07:42:40.377 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-23T07:42:40.377 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-23T07:42:40.380 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi105.front.sepia.ceph.com to complete... 2024-03-23T07:42:43.040 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-23T07:42:43.273 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-23T07:42:43.356 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T07:42:44.502 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T07:42:44.503 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-23T07:42:44.900 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T07:42:44.900 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:42:44.900 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-03-23T07:42:44.900 INFO:teuthology.orchestra.run.smithi132.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-23T07:42:44.900 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:42:44.900 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-23T07:42:45.184 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.1... 2024-03-23T07:42:45.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T07:42:45.401 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-23T07:42:45.401 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-23T07:42:45.403 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi132.front.sepia.ceph.com to complete... 2024-03-23T07:42:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23 07:42:53 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-23T07:42:53.968 INFO:teuthology.orchestra.run.smithi105.stderr:FINISHED --2024-03-23 07:42:53-- 2024-03-23T07:42:53.968 INFO:teuthology.orchestra.run.smithi105.stderr:Total wall clock time: 14s 2024-03-23T07:42:53.968 INFO:teuthology.orchestra.run.smithi105.stderr:Downloaded: 1 files, 321M in 13s (24.2 MB/s) 2024-03-23T07:42:53.971 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-23T07:42:53.971 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-23T07:42:55.903 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying... ######################################## 2024-03-23T07:42:55.954 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing... ######################################## 2024-03-23T07:42:56.033 INFO:teuthology.orchestra.run.smithi105.stdout:Updating / installing... 2024-03-23T07:43:03.520 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-23T07:43:14.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23 07:43:14 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-23T07:43:14.705 INFO:teuthology.orchestra.run.smithi132.stderr:FINISHED --2024-03-23 07:43:14-- 2024-03-23T07:43:14.705 INFO:teuthology.orchestra.run.smithi132.stderr:Total wall clock time: 29s 2024-03-23T07:43:14.705 INFO:teuthology.orchestra.run.smithi132.stderr:Downloaded: 1 files, 321M in 29s (11.2 MB/s) 2024-03-23T07:43:14.708 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.1... 2024-03-23T07:43:14.708 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-23T07:43:16.637 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying... ######################################## 2024-03-23T07:43:16.691 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing... ######################################## 2024-03-23T07:43:16.771 INFO:teuthology.orchestra.run.smithi132.stdout:Updating / installing... 2024-03-23T07:43:19.688 INFO:teuthology.orchestra.run.smithi105.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T07:43:24.245 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-23T07:43:38.677 INFO:teuthology.orchestra.run.smithi132.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T07:43:42.192 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-23T07:43:42.192 DEBUG:teuthology.orchestra.run.smithi105:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-23T07:43:42.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-23T07:43:42.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-23T07:43:42.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-23T07:43:42.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-23T07:43:42.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-23T07:43:42.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-23T07:43:42.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-23T07:43:42.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-23T07:43:42.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-23T07:43:42.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-23T07:43:42.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-23T07:43:42.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-23T07:43:42.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-23T07:43:42.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-23T07:43:42.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-23T07:43:42.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-23T07:43:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-23T07:43:42.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-23T07:43:42.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-23T07:43:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-23T07:43:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-23T07:43:42.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-23T07:43:42.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-23T07:43:42.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-23T07:43:42.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-23T07:43:42.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-23T07:43:42.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-23T07:43:42.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-23T07:43:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-23T07:43:42.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-23T07:43:42.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-23T07:43:42.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-23T07:43:42.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-23T07:43:42.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-23T07:43:42.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-23T07:43:42.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-23T07:43:42.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-23T07:43:42.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-23T07:43:42.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-23T07:43:42.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-23T07:43:42.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-23T07:43:42.292 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-23T07:43:42.293 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-23T07:43:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-23T07:43:42.295 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-23T07:43:42.296 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-23T07:43:42.297 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-23T07:43:42.298 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-23T07:43:42.299 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-23T07:43:42.300 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-23T07:43:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-23T07:43:42.302 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-23T07:43:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-23T07:43:42.304 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-23T07:43:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-23T07:43:42.306 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-23T07:43:42.307 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-23T07:43:42.308 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-23T07:43:42.309 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-23T07:43:42.310 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-23T07:43:42.311 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-23T07:43:42.312 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-23T07:43:42.313 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-23T07:43:42.314 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-23T07:43:42.315 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-23T07:43:42.316 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-23T07:43:42.317 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-23T07:43:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-23T07:43:42.319 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-23T07:43:42.320 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-23T07:43:42.321 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-23T07:43:42.322 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-23T07:43:42.323 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-23T07:43:42.324 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-23T07:43:42.325 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-23T07:43:42.326 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-23T07:43:42.327 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-23T07:43:42.328 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-23T07:43:42.329 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-23T07:43:42.330 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-23T07:43:42.331 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-23T07:43:42.332 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-23T07:43:42.333 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-23T07:43:42.334 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-23T07:43:42.335 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-23T07:43:42.336 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-23T07:43:42.337 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-23T07:43:42.338 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-23T07:43:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-23T07:43:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-23T07:43:42.341 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-23T07:43:42.342 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-23T07:43:42.343 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-23T07:43:42.344 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-23T07:43:42.345 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-23T07:43:42.346 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-23T07:43:42.347 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-23T07:43:42.348 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-23T07:43:42.349 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-23T07:43:42.350 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-23T07:43:42.351 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-23T07:43:42.352 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-23T07:43:42.353 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-23T07:43:42.354 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-23T07:43:42.355 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-23T07:43:42.357 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-23T07:43:42.358 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-23T07:43:42.359 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-23T07:43:42.360 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-23T07:43:42.361 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-23T07:43:42.362 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-23T07:43:42.363 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-23T07:43:42.364 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-23T07:43:42.365 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-23T07:43:42.366 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-23T07:43:42.367 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-23T07:43:42.368 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-23T07:43:42.369 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-23T07:43:42.370 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-23T07:43:42.371 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-23T07:43:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-23T07:43:42.373 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-23T07:43:42.374 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-23T07:43:42.375 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-23T07:43:42.376 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-23T07:43:42.377 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-23T07:43:42.378 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-23T07:43:42.379 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-23T07:43:42.380 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-23T07:43:42.381 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-23T07:43:42.382 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-23T07:43:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-23T07:43:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-23T07:43:42.385 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-23T07:43:42.386 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T07:43:42.390 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-23T07:43:42.390 DEBUG:teuthology.orchestra.run.smithi105:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-23T07:43:42.412 INFO:teuthology.orchestra.run.smithi105.stdout:postinstall scriptlet (using /bin/sh): 2024-03-23T07:43:42.412 INFO:teuthology.orchestra.run.smithi105.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:fi 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:for file in vmlinuz System.map config; do 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: fi 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:done 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:fi 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-23T07:43:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-23T07:43:42.414 INFO:teuthology.orchestra.run.smithi105.stdout:fi 2024-03-23T07:43:42.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 68.el9 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8264254 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-23T07:43:42.499 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:43:42.500 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T07:43:42.501 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T07:43:42.501 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 6.8.0-g5ec3e6e88b2e 2024-03-23T07:43:42.501 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T07:43:43.023 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-03-23T07:43:44.410 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-23T07:43:44.446 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-03-23T07:43:44.456 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-03-23T07:43:44.476 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.zj3Tr2CJnI 2024-03-23T07:43:44.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zj3Tr2CJnI 2024-03-23T07:43:44.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.zj3Tr2CJnI 2024-03-23T07:43:44.684 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.zj3Tr2CJnI is 6KB 2024-03-23T07:43:44.704 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.zj3Tr2CJnI 2024-03-23T07:43:44.718 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T07:43:44.782 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-23T07:43:44.782 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-23T07:43:44.782 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-23T07:43:44.783 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-23T07:43:44.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-23T07:43:44.933 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-03-23T07:44:01.328 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi132.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-23T07:44:01.328 DEBUG:teuthology.orchestra.run.smithi132:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-23T07:44:01.401 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-23T07:44:01.401 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-23T07:44:01.401 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-23T07:44:01.401 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-23T07:44:01.401 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-23T07:44:01.401 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-23T07:44:01.401 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-23T07:44:01.402 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-23T07:44:01.403 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-23T07:44:01.404 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-23T07:44:01.405 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-23T07:44:01.406 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-23T07:44:01.407 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-23T07:44:01.408 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-23T07:44:01.409 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-23T07:44:01.410 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-23T07:44:01.411 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-23T07:44:01.412 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-23T07:44:01.413 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-23T07:44:01.414 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-23T07:44:01.415 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-23T07:44:01.416 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-23T07:44:01.417 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-23T07:44:01.418 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-23T07:44:01.419 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-23T07:44:01.420 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-23T07:44:01.421 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-23T07:44:01.422 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-23T07:44:01.423 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-23T07:44:01.424 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-23T07:44:01.425 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-23T07:44:01.426 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-23T07:44:01.427 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-23T07:44:01.428 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-23T07:44:01.429 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-23T07:44:01.430 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-23T07:44:01.431 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-23T07:44:01.432 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-23T07:44:01.433 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-23T07:44:01.434 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-23T07:44:01.435 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-23T07:44:01.436 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-23T07:44:01.437 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-23T07:44:01.438 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-23T07:44:01.439 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-23T07:44:01.440 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-23T07:44:01.441 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-23T07:44:01.442 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-23T07:44:01.443 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-23T07:44:01.444 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-23T07:44:01.445 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-23T07:44:01.446 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-23T07:44:01.447 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-23T07:44:01.448 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-23T07:44:01.449 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-23T07:44:01.450 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-23T07:44:01.451 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-23T07:44:01.452 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-23T07:44:01.453 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-23T07:44:01.454 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-23T07:44:01.455 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-23T07:44:01.456 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-23T07:44:01.457 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-23T07:44:01.458 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-23T07:44:01.459 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-23T07:44:01.460 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-23T07:44:01.461 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-23T07:44:01.462 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-23T07:44:01.463 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-23T07:44:01.464 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-23T07:44:01.465 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-23T07:44:01.466 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-23T07:44:01.467 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-23T07:44:01.468 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-23T07:44:01.469 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-23T07:44:01.470 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-23T07:44:01.471 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-23T07:44:01.472 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-23T07:44:01.473 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-23T07:44:01.474 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-23T07:44:01.475 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-23T07:44:01.476 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-23T07:44:01.477 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-23T07:44:01.478 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-23T07:44:01.479 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-23T07:44:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-23T07:44:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-23T07:44:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-23T07:44:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-23T07:44:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-23T07:44:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-23T07:44:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-23T07:44:01.487 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-23T07:44:01.488 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-23T07:44:01.489 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-23T07:44:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-23T07:44:01.491 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-23T07:44:01.492 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-23T07:44:01.493 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-23T07:44:01.494 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-23T07:44:01.495 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-23T07:44:01.496 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-23T07:44:01.497 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-23T07:44:01.498 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-23T07:44:01.499 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-23T07:44:01.500 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-23T07:44:01.501 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-23T07:44:01.502 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-23T07:44:01.503 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-23T07:44:01.504 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-23T07:44:01.505 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-23T07:44:01.506 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-23T07:44:01.507 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-23T07:44:01.508 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-23T07:44:01.509 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-23T07:44:01.510 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-23T07:44:01.511 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-23T07:44:01.512 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-23T07:44:01.513 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-23T07:44:01.514 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-23T07:44:01.515 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-23T07:44:01.516 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-23T07:44:01.517 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-23T07:44:01.518 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-23T07:44:01.519 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-23T07:44:01.520 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-23T07:44:01.521 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-23T07:44:01.522 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-23T07:44:01.523 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-23T07:44:01.524 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-23T07:44:01.525 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-23T07:44:01.526 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-23T07:44:01.527 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-23T07:44:01.528 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-23T07:44:01.529 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-23T07:44:01.530 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-23T07:44:01.531 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-23T07:44:01.532 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-23T07:44:01.533 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T07:44:01.537 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-23T07:44:01.537 DEBUG:teuthology.orchestra.run.smithi132:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-23T07:44:01.559 INFO:teuthology.orchestra.run.smithi132.stdout:postinstall scriptlet (using /bin/sh): 2024-03-23T07:44:01.559 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-23T07:44:01.559 INFO:teuthology.orchestra.run.smithi132.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T07:44:01.559 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:for file in vmlinuz System.map config; do 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout: fi 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:done 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-23T07:44:01.560 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-23T07:44:01.561 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-23T07:44:01.561 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-03-23T07:44:01.562 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -qi grub2-tools 2024-03-23T07:44:01.644 INFO:teuthology.orchestra.run.smithi132.stdout:Name : grub2-tools 2024-03-23T07:44:01.644 INFO:teuthology.orchestra.run.smithi132.stdout:Epoch : 1 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Version : 2.06 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Release : 68.el9 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: x86_64 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Group : Unspecified 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Size : 8264254 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:License : GPLv3+ 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Packager : builder@centos.org 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:Vendor : CentOS 2024-03-23T07:44:01.645 INFO:teuthology.orchestra.run.smithi132.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout:Summary : Support tools for GRUB. 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout:Description : 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout:hardware devices. 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:44:01.646 INFO:teuthology.orchestra.run.smithi132.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T07:44:01.647 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T07:44:01.647 INFO:teuthology.task.kernel:Updating grub on smithi132 to boot 6.8.0-g5ec3e6e88b2e 2024-03-23T07:44:01.647 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T07:44:02.246 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2024-03-23T07:44:03.647 INFO:teuthology.orchestra.run.smithi132.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-23T07:44:03.682 INFO:teuthology.orchestra.run.smithi132.stderr:done 2024-03-23T07:44:03.685 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-03-23T07:44:03.699 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.5n29ca5P2i 2024-03-23T07:44:03.700 DEBUG:teuthology.orchestra.run.smithi132:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5n29ca5P2i 2024-03-23T07:44:03.768 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.5n29ca5P2i 2024-03-23T07:44:03.904 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.5n29ca5P2i is 6KB 2024-03-23T07:44:03.952 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.5n29ca5P2i 2024-03-23T07:44:03.966 DEBUG:teuthology.orchestra.run.smithi132:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T07:44:04.030 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-23T07:44:04.030 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-23T07:44:04.031 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-23T07:44:04.031 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-23T07:44:04.032 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-23T07:44:04.183 DEBUG:teuthology.orchestra.run.smithi132:> sudo shutdown -r now 2024-03-23T07:44:14.946 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-23T07:44:14.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-23T07:44:14.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:44:33.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-03-23T07:44:34.187 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-23T07:44:34.187 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-23T07:44:34.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:44:42.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-23T07:44:42.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:44:45.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-03-23T07:44:52.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-03-23T07:44:57.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-23T07:44:57.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:45:01.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-23T07:45:01.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:45:04.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-03-23T07:45:16.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-23T07:45:16.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:45:29.243 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-23T07:45:29.807 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-23T07:45:29.807 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g5ec3e6e88b2e"... 2024-03-23T07:45:29.808 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-03-23T07:45:29.846 INFO:teuthology.orchestra.run.smithi105.stdout:6.8.0-g5ec3e6e88b2e 2024-03-23T07:45:29.847 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-23T07:45:29.847 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T07:45:29.847 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T07:45:30.848 DEBUG:teuthology.parallel:result is None 2024-03-23T07:45:48.452 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-03-23T07:45:49.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-23T07:45:49.152 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-g5ec3e6e88b2e"... 2024-03-23T07:45:49.152 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-03-23T07:45:49.192 INFO:teuthology.orchestra.run.smithi132.stdout:6.8.0-g5ec3e6e88b2e 2024-03-23T07:45:49.192 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-23T07:45:49.192 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T07:45:49.192 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T07:45:50.193 DEBUG:teuthology.parallel:result is None 2024-03-23T07:45:50.194 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T07:45:50.206 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T07:45:50.206 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T07:45:50.215 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T07:45:50.264 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T07:45:50.271 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T07:45:50.277 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T07:45:50.277 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T07:45:50.282 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T07:45:50.359 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T07:45:50.365 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T07:45:50.366 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T07:45:50.373 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T07:45:50.507 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T07:45:50.545 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T07:45:50.606 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T07:45:50.612 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T07:45:50.617 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T07:45:50.626 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T07:45:50.626 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T07:45:50.631 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T07:45:50.722 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T07:45:50.732 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T07:45:50.732 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T07:45:50.738 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T07:45:50.783 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T07:45:50.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T07:45:50.972 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T07:45:51.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T07:45:51.151 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:45:51.151 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T07:45:51.244 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T07:45:51.323 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T07:45:51.404 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T07:45:51.470 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T07:45:51.560 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:45:51.560 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T07:45:51.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-03-23T07:45:51.686 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-03-23T07:45:51.800 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T07:45:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T07:45:52.214 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T07:45:52.222 INFO:teuthology.task.internal:Starting timer... 2024-03-23T07:45:52.222 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T07:45:52.233 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T07:45:52.242 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-03-23T07:45:52.356 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-03-23T07:45:52.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-03-23T07:45:52.483 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-03-23T07:45:52.486 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T07:45:52.487 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-03-23T07:45:52.554 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-03-23T07:45:52.554 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-03-23T07:45:52.608 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-03-23T07:45:52.608 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive'} 2024-03-23T07:45:52.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T07:45:52.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:45:52.685 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T07:45:52.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:45:52.767 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T07:45:52.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-03-23T07:45:52.851 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-03-23T07:45:52.962 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T07:45:52.971 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-23T07:45:52.976 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-23T07:45:53.095 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T07:45:53.111 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T07:45:53.112 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi132.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T07:51:55.766 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com')] 2024-03-23T07:51:55.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-23T07:51:55.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:51:55.917 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-23T07:51:56.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-23T07:51:56.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-23T07:51:56.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:51:56.211 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-03-23T07:51:56.360 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-23T07:51:56.361 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T07:51:56.376 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T07:51:56.376 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T07:51:56.377 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T07:51:56.384 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T07:51:56.384 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T07:51:56.479 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T07:51:56.481 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T07:51:56.532 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T07:51:56.536 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T07:51:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-03-23T07:51:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-03-23T07:51:56.667 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-03-23T07:51:56.670 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-03-23T07:51:56.719 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T07:51:56.720 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T07:51:56.773 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T07:51:56.775 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T07:51:56.895 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-23T07:51:56.902 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-23T07:51:56.904 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T07:51:56.904 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-23T07:51:56.904 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.904 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.904 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.905 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.909 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T07:51:56.909 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-23T07:51:56.909 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.910 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.910 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.910 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:51:56.911 INFO:teuthology.run_tasks:Running task install... 2024-03-23T07:51:56.919 DEBUG:teuthology.task.install:project ceph 2024-03-23T07:51:56.919 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}} 2024-03-23T07:51:56.919 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'} 2024-03-23T07:51:56.919 INFO:teuthology.task.install:Using flavor: default 2024-03-23T07:51:56.925 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T07:51:56.925 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-23T07:51:56.925 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T07:51:56.926 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T07:51:56.928 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T07:51:56.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T07:51:57.073 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/ 2024-03-23T07:51:57.073 INFO:teuthology.task.install.rpm:Package version is 18.2.2-954.g82316b04 2024-03-23T07:51:57.142 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/ 2024-03-23T07:51:57.142 INFO:teuthology.task.install.rpm:Package version is 18.2.2-954.g82316b04 2024-03-23T07:51:57.246 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T07:51:57.247 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:51:57.247 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T07:51:57.311 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T07:51:57.312 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:51:57.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T07:51:57.341 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T07:51:57.342 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82316b04821a6c2c1d458a881920454bbadb8837/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T07:51:57.411 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T07:51:57.411 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82316b04821a6c2c1d458a881920454bbadb8837/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T07:51:57.480 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T07:51:57.549 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T07:51:57.614 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T07:51:57.678 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T07:51:57.702 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-03-23T07:51:57.706 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-03-23T07:51:57.763 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-03-23T07:51:57.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-03-23T07:51:58.067 INFO:teuthology.orchestra.run.smithi132.stdout:74 files removed 2024-03-23T07:51:58.107 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T07:51:58.130 INFO:teuthology.orchestra.run.smithi105.stdout:74 files removed 2024-03-23T07:51:58.169 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T07:51:59.121 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 173 kB/s | 77 kB 00:00 2024-03-23T07:51:59.165 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 139 kB/s | 77 kB 00:00 2024-03-23T07:51:59.705 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 20 kB/s | 9.9 kB 00:00 2024-03-23T07:51:59.766 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 20 kB/s | 9.9 kB 00:00 2024-03-23T07:52:00.197 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-03-23T07:52:00.333 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-03-23T07:52:01.442 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 5.7 MB/s | 6.7 MB 00:01 2024-03-23T07:52:01.486 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 6.2 MB/s | 6.7 MB 00:01 2024-03-23T07:52:03.126 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-03-23T07:52:03.237 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-03-23T07:52:04.213 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-23T07:52:04.491 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-03-23T07:52:07.765 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-03-23T07:52:12.791 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.1 MB 00:01 2024-03-23T07:52:14.516 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-03-23T07:52:15.584 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-23T07:52:21.043 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-23T07:52:21.460 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-03-23T07:52:24.063 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T07:52:24.065 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T07:52:24.085 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T07:52:24.086 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T07:52:24.088 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T07:52:24.089 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T07:52:24.092 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T07:52:24.095 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T07:52:24.097 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T07:52:24.182 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-23T07:52:24.203 INFO:teuthology.orchestra.run.smithi105.stdout:=========================================================================================== 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout:=========================================================================================== 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:18.2.2-954.g82316b04.el9 ceph 6.5 k 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:18.2.2-954.g82316b04.el9 ceph 5.1 M 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 843 k 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-954.g82316b04.el9 ceph 143 k 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:18.2.2-954.g82316b04.el9 ceph 1.4 M 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 138 k 2024-03-23T07:52:24.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 1.7 M 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 7.4 M 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 49 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 7.7 M 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:18.2.2-954.g82316b04.el9 ceph 40 M 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 264 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 224 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 221 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 18 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 31 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 704 k 2024-03-23T07:52:24.205 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:18.2.2-954.g82316b04.el9 ceph 161 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:18.2.2-954.g82316b04.el9 ceph 320 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 297 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 99 k 2024-03-23T07:52:24.206 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 87 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 172 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T07:52:24.207 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.3 M 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T07:52:24.208 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 18 M 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 23 k 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:18.2.2-954.g82316b04.el9 ceph 2.1 M 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 245 k 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.7 M 2024-03-23T07:52:24.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:18.2.2-954.g82316b04.el9 ceph 16 M 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 15 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:18.2.2-954.g82316b04.el9 ceph 25 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T07:52:24.210 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:18.2.2-954.g82316b04.el9 ceph 166 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 474 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.5 M 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T07:52:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:18.2.2-954.g82316b04.el9 ceph 45 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T07:52:24.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T07:52:24.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T07:52:24.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T07:52:24.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout:=========================================================================================== 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout:Install 132 Packages 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 24 Packages 2024-03-23T07:52:24.216 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:52:24.217 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 204 M 2024-03-23T07:52:24.217 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-23T07:52:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:(1/156): ceph-18.2.2-954.g82316b04.el9.x86_64.r 16 kB/s | 6.5 kB 00:00 2024-03-23T07:52:25.644 INFO:teuthology.orchestra.run.smithi105.stdout:(2/156): ceph-fuse-18.2.2-954.g82316b04.el9.x86 2.3 MB/s | 843 kB 00:00 2024-03-23T07:52:25.803 INFO:teuthology.orchestra.run.smithi105.stdout:(3/156): ceph-base-18.2.2-954.g82316b04.el9.x86 5.5 MB/s | 5.1 MB 00:00 2024-03-23T07:52:25.836 INFO:teuthology.orchestra.run.smithi105.stdout:(4/156): ceph-immutable-object-cache-18.2.2-954 747 kB/s | 143 kB 00:00 2024-03-23T07:52:26.062 INFO:teuthology.orchestra.run.smithi105.stdout:(5/156): ceph-mds-18.2.2-954.g82316b04.el9.x86_ 8.2 MB/s | 2.1 MB 00:00 2024-03-23T07:52:26.212 INFO:teuthology.orchestra.run.smithi105.stdout:(6/156): ceph-mgr-18.2.2-954.g82316b04.el9.x86_ 3.9 MB/s | 1.4 MB 00:00 2024-03-23T07:52:26.579 INFO:teuthology.orchestra.run.smithi105.stdout:(7/156): ceph-mon-18.2.2-954.g82316b04.el9.x86_ 9.1 MB/s | 4.7 MB 00:00 2024-03-23T07:52:26.921 INFO:teuthology.orchestra.run.smithi105.stdout:(8/156): ceph-common-18.2.2-954.g82316b04.el9.x 9.0 MB/s | 18 MB 00:02 2024-03-23T07:52:26.971 INFO:teuthology.orchestra.run.smithi105.stdout:(9/156): ceph-selinux-18.2.2-954.g82316b04.el9. 514 kB/s | 25 kB 00:00 2024-03-23T07:52:27.255 INFO:teuthology.orchestra.run.smithi105.stdout:(10/156): ceph-radosgw-18.2.2-954.g82316b04.el9 11 MB/s | 7.7 MB 00:00 2024-03-23T07:52:27.330 INFO:teuthology.orchestra.run.smithi105.stdout:(11/156): cephfs-mirror-18.2.2-954.g82316b04.el 2.9 MB/s | 221 kB 00:00 2024-03-23T07:52:27.405 INFO:teuthology.orchestra.run.smithi105.stdout:(12/156): libcephfs-devel-18.2.2-954.g82316b04. 418 kB/s | 31 kB 00:00 2024-03-23T07:52:27.539 INFO:teuthology.orchestra.run.smithi105.stdout:(13/156): libcephfs2-18.2.2-954.g82316b04.el9.x 5.2 MB/s | 704 kB 00:00 2024-03-23T07:52:27.631 INFO:teuthology.orchestra.run.smithi105.stdout:(14/156): libcephsqlite-18.2.2-954.g82316b04.el 1.8 MB/s | 166 kB 00:00 2024-03-23T07:52:27.715 INFO:teuthology.orchestra.run.smithi105.stdout:(15/156): librados-devel-18.2.2-954.g82316b04.e 1.5 MB/s | 127 kB 00:00 2024-03-23T07:52:27.806 INFO:teuthology.orchestra.run.smithi105.stdout:(16/156): libradosstriper1-18.2.2-954.g82316b04 5.1 MB/s | 474 kB 00:00 2024-03-23T07:52:28.232 INFO:teuthology.orchestra.run.smithi105.stdout:(17/156): librgw2-18.2.2-954.g82316b04.el9.x86_ 11 MB/s | 4.5 MB 00:00 2024-03-23T07:52:28.299 INFO:teuthology.orchestra.run.smithi105.stdout:(18/156): python3-ceph-argparse-18.2.2-954.g823 685 kB/s | 45 kB 00:00 2024-03-23T07:52:28.365 INFO:teuthology.orchestra.run.smithi105.stdout:(19/156): python3-ceph-common-18.2.2-954.g82316 1.9 MB/s | 127 kB 00:00 2024-03-23T07:52:28.432 INFO:teuthology.orchestra.run.smithi105.stdout:(20/156): python3-cephfs-18.2.2-954.g82316b04.e 2.4 MB/s | 161 kB 00:00 2024-03-23T07:52:28.658 INFO:teuthology.orchestra.run.smithi105.stdout:(21/156): ceph-osd-18.2.2-954.g82316b04.el9.x86 6.7 MB/s | 16 MB 00:02 2024-03-23T07:52:28.700 INFO:teuthology.orchestra.run.smithi105.stdout:(22/156): python3-rados-18.2.2-954.g82316b04.el 1.2 MB/s | 320 kB 00:00 2024-03-23T07:52:28.758 INFO:teuthology.orchestra.run.smithi105.stdout:(23/156): python3-rgw-18.2.2-954.g82316b04.el9. 1.7 MB/s | 99 kB 00:00 2024-03-23T07:52:28.800 INFO:teuthology.orchestra.run.smithi105.stdout:(24/156): python3-rbd-18.2.2-954.g82316b04.el9. 2.1 MB/s | 297 kB 00:00 2024-03-23T07:52:28.826 INFO:teuthology.orchestra.run.smithi105.stdout:(25/156): rbd-fuse-18.2.2-954.g82316b04.el9.x86 1.3 MB/s | 87 kB 00:00 2024-03-23T07:52:28.901 INFO:teuthology.orchestra.run.smithi105.stdout:(26/156): rbd-nbd-18.2.2-954.g82316b04.el9.x86_ 2.3 MB/s | 172 kB 00:00 2024-03-23T07:52:28.968 INFO:teuthology.orchestra.run.smithi105.stdout:(27/156): ceph-grafana-dashboards-18.2.2-954.g8 356 kB/s | 23 kB 00:00 2024-03-23T07:52:29.043 INFO:teuthology.orchestra.run.smithi105.stdout:(28/156): ceph-mgr-cephadm-18.2.2-954.g82316b04 1.8 MB/s | 138 kB 00:00 2024-03-23T07:52:29.252 INFO:teuthology.orchestra.run.smithi105.stdout:(29/156): ceph-mgr-dashboard-18.2.2-954.g82316b 8.4 MB/s | 1.7 MB 00:00 2024-03-23T07:52:29.327 INFO:teuthology.orchestra.run.smithi105.stdout:(30/156): rbd-mirror-18.2.2-954.g82316b04.el9.x 5.7 MB/s | 3.0 MB 00:00 2024-03-23T07:52:29.427 INFO:teuthology.orchestra.run.smithi105.stdout:(31/156): ceph-mgr-modules-core-18.2.2-954.g823 2.4 MB/s | 245 kB 00:00 2024-03-23T07:52:29.544 INFO:teuthology.orchestra.run.smithi105.stdout:(32/156): ceph-mgr-rook-18.2.2-954.g82316b04.el 425 kB/s | 49 kB 00:00 2024-03-23T07:52:29.661 INFO:teuthology.orchestra.run.smithi105.stdout:(33/156): ceph-prometheus-alerts-18.2.2-954.g82 128 kB/s | 15 kB 00:00 2024-03-23T07:52:29.869 INFO:teuthology.orchestra.run.smithi105.stdout:(34/156): ceph-mgr-diskprediction-local-18.2.2- 12 MB/s | 7.4 MB 00:00 2024-03-23T07:52:29.903 INFO:teuthology.orchestra.run.smithi105.stdout:(35/156): ceph-volume-18.2.2-954.g82316b04.el9. 1.1 MB/s | 264 kB 00:00 2024-03-23T07:52:29.937 INFO:teuthology.orchestra.run.smithi105.stdout:(36/156): cephadm-18.2.2-954.g82316b04.el9.noar 3.3 MB/s | 224 kB 00:00 2024-03-23T07:52:30.029 INFO:teuthology.orchestra.run.smithi105.stdout:(37/156): cephfs-top-18.2.2-954.g82316b04.el9.n 146 kB/s | 18 kB 00:00 2024-03-23T07:52:30.463 INFO:teuthology.orchestra.run.smithi105.stdout:(38/156): ceph-test-18.2.2-954.g82316b04.el9.x8 11 MB/s | 40 MB 00:03 2024-03-23T07:52:30.597 INFO:teuthology.orchestra.run.smithi105.stdout:(39/156): bison-3.7.4-5.el9.x86_64.rpm 1.6 MB/s | 920 kB 00:00 2024-03-23T07:52:30.638 INFO:teuthology.orchestra.run.smithi105.stdout:(40/156): automake-1.16.2-8.el9.noarch.rpm 946 kB/s | 662 kB 00:00 2024-03-23T07:52:30.663 INFO:teuthology.orchestra.run.smithi105.stdout:(41/156): boost-program-options-1.75.0-8.el9.x8 538 kB/s | 106 kB 00:00 2024-03-23T07:52:30.697 INFO:teuthology.orchestra.run.smithi105.stdout:(42/156): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 307 kB 00:00 2024-03-23T07:52:30.722 INFO:teuthology.orchestra.run.smithi105.stdout:(43/156): flexiblas-3.0.4-8.el9.x86_64.rpm 576 kB/s | 32 kB 00:00 2024-03-23T07:52:30.872 INFO:teuthology.orchestra.run.smithi105.stdout:(44/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-03-23T07:52:30.898 INFO:teuthology.orchestra.run.smithi105.stdout:(45/156): flexiblas-openblas-openmp-3.0.4-8.el9 88 kB/s | 17 kB 00:00 2024-03-23T07:52:30.923 INFO:teuthology.orchestra.run.smithi105.stdout:(46/156): libaio-devel-0.3.111-13.el9.x86_64.rp 52 kB/s | 10 kB 00:00 2024-03-23T07:52:30.957 INFO:teuthology.orchestra.run.smithi105.stdout:(47/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-23T07:52:30.990 INFO:teuthology.orchestra.run.smithi105.stdout:(48/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 683 kB/s | 44 kB 00:00 2024-03-23T07:52:31.023 INFO:teuthology.orchestra.run.smithi105.stdout:(49/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.6 MB/s | 661 kB 00:00 2024-03-23T07:52:31.057 INFO:teuthology.orchestra.run.smithi105.stdout:(50/156): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2024-03-23T07:52:31.091 INFO:teuthology.orchestra.run.smithi105.stdout:(51/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.5 MB/s | 282 kB 00:00 2024-03-23T07:52:31.141 INFO:teuthology.orchestra.run.smithi105.stdout:(52/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 414 kB/s | 46 kB 00:00 2024-03-23T07:52:31.158 INFO:teuthology.orchestra.run.smithi105.stdout:(53/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.2 MB/s | 217 kB 00:00 2024-03-23T07:52:31.183 INFO:teuthology.orchestra.run.smithi105.stdout:(54/156): openblas-0.3.21-2.el9.x86_64.rpm 383 kB/s | 34 kB 00:00 2024-03-23T07:52:31.317 INFO:teuthology.orchestra.run.smithi105.stdout:(55/156): postgresql-13.11-1.el9.x86_64.rpm 9.7 MB/s | 1.5 MB 00:00 2024-03-23T07:52:31.434 INFO:teuthology.orchestra.run.smithi105.stdout:(56/156): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-03-23T07:52:31.467 INFO:teuthology.orchestra.run.smithi105.stdout:(57/156): postgresql-contrib-13.11-1.el9.x86_64 2.8 MB/s | 811 kB 00:00 2024-03-23T07:52:31.492 INFO:teuthology.orchestra.run.smithi105.stdout:(58/156): postgresql-private-libs-13.11-1.el9.x 823 kB/s | 134 kB 00:00 2024-03-23T07:52:31.526 INFO:teuthology.orchestra.run.smithi105.stdout:(59/156): python3-jsonpatch-1.21-16.el9.noarch. 774 kB/s | 24 kB 00:00 2024-03-23T07:52:31.701 INFO:teuthology.orchestra.run.smithi105.stdout:(60/156): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-23T07:52:31.751 INFO:teuthology.orchestra.run.smithi105.stdout:(61/156): python3-jsonpointer-2.0-4.el9.noarch. 83 kB/s | 18 kB 00:00 2024-03-23T07:52:31.868 INFO:teuthology.orchestra.run.smithi105.stdout:(62/156): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-03-23T07:52:31.893 INFO:teuthology.orchestra.run.smithi105.stdout:(63/156): python3-mako-1.1.4-6.el9.noarch.rpm 798 kB/s | 144 kB 00:00 2024-03-23T07:52:31.919 INFO:teuthology.orchestra.run.smithi105.stdout:(64/156): python3-markupsafe-1.1.1-12.el9.x86_6 202 kB/s | 32 kB 00:00 2024-03-23T07:52:31.952 INFO:teuthology.orchestra.run.smithi105.stdout:(65/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 217 kB 00:00 2024-03-23T07:52:32.086 INFO:teuthology.orchestra.run.smithi105.stdout:(66/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-23T07:52:32.111 INFO:teuthology.orchestra.run.smithi105.stdout:(67/156): python3-oauthlib-3.1.1-5.el9.noarch.r 926 kB/s | 174 kB 00:00 2024-03-23T07:52:32.136 INFO:teuthology.orchestra.run.smithi105.stdout:(68/156): python3-packaging-20.9-5.el9.noarch.r 410 kB/s | 69 kB 00:00 2024-03-23T07:52:32.178 INFO:teuthology.orchestra.run.smithi105.stdout:(69/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-03-23T07:52:32.204 INFO:teuthology.orchestra.run.smithi105.stdout:(70/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-23T07:52:32.229 INFO:teuthology.orchestra.run.smithi105.stdout:(71/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2024-03-23T07:52:32.295 INFO:teuthology.orchestra.run.smithi105.stdout:(72/156): python3-pytz-2021.1-5.el9.noarch.rpm 724 kB/s | 47 kB 00:00 2024-03-23T07:52:32.321 INFO:teuthology.orchestra.run.smithi105.stdout:(73/156): python3-requests-oauthlib-1.3.0-12.el 473 kB/s | 43 kB 00:00 2024-03-23T07:52:32.638 INFO:teuthology.orchestra.run.smithi105.stdout:(74/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-23T07:52:32.663 INFO:teuthology.orchestra.run.smithi105.stdout:(75/156): python3-toml-0.10.2-6.el9.noarch.rpm 119 kB/s | 39 kB 00:00 2024-03-23T07:52:32.688 INFO:teuthology.orchestra.run.smithi105.stdout:(76/156): socat-1.7.4.1-5.el9.x86_64.rpm 857 kB/s | 300 kB 00:00 2024-03-23T07:52:32.730 INFO:teuthology.orchestra.run.smithi105.stdout:(77/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-23T07:52:32.755 INFO:teuthology.orchestra.run.smithi105.stdout:(78/156): uuid-1.6.2-55.el9.x86_64.rpm 859 kB/s | 56 kB 00:00 2024-03-23T07:52:32.780 INFO:teuthology.orchestra.run.smithi105.stdout:(79/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 488 kB/s | 44 kB 00:00 2024-03-23T07:52:32.790 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 785 kB/s | 6.1 MB 00:07 2024-03-23T07:52:32.797 INFO:teuthology.orchestra.run.smithi105.stdout:(80/156): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-03-23T07:52:32.839 INFO:teuthology.orchestra.run.smithi105.stdout:(81/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-23T07:52:32.873 INFO:teuthology.orchestra.run.smithi105.stdout:(82/156): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-23T07:52:32.898 INFO:teuthology.orchestra.run.smithi105.stdout:(83/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-23T07:52:32.932 INFO:teuthology.orchestra.run.smithi105.stdout:(84/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-23T07:52:32.957 INFO:teuthology.orchestra.run.smithi105.stdout:(85/156): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-23T07:52:34.899 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-03-23T07:52:39.301 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 4.9 MB/s | 21 MB 00:04 2024-03-23T07:52:45.117 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-23T07:52:47.656 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T07:52:47.657 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T07:52:47.678 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T07:52:47.679 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T07:52:47.680 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T07:52:47.682 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T07:52:47.684 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T07:52:47.688 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T07:52:47.690 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T07:52:47.774 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-23T07:52:47.797 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================== 2024-03-23T07:52:47.797 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================== 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:18.2.2-954.g82316b04.el9 ceph 6.5 k 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:18.2.2-954.g82316b04.el9 ceph 5.1 M 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 843 k 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-954.g82316b04.el9 ceph 143 k 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:18.2.2-954.g82316b04.el9 ceph 1.4 M 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 138 k 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 1.7 M 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 7.4 M 2024-03-23T07:52:47.798 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 49 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 7.7 M 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:18.2.2-954.g82316b04.el9 ceph 40 M 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 264 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 224 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 221 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 18 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 31 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 704 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T07:52:47.799 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:18.2.2-954.g82316b04.el9 ceph 161 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:18.2.2-954.g82316b04.el9 ceph 320 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 297 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:18.2.2-954.g82316b04.el9 ceph 99 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:18.2.2-954.g82316b04.el9 ceph 87 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:18.2.2-954.g82316b04.el9 ceph 172 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T07:52:47.800 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T07:52:47.801 INFO:teuthology.orchestra.run.smithi132.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.3 M 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 3.0 M 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T07:52:47.802 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 18 M 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 23 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:18.2.2-954.g82316b04.el9 ceph 2.1 M 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 245 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.7 M 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:18.2.2-954.g82316b04.el9 ceph 16 M 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:18.2.2-954.g82316b04.el9 ceph-noarch 15 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:18.2.2-954.g82316b04.el9 ceph 25 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T07:52:47.803 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:18.2.2-954.g82316b04.el9 ceph 166 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:18.2.2-954.g82316b04.el9 ceph 474 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:18.2.2-954.g82316b04.el9 ceph 4.5 M 2024-03-23T07:52:47.804 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:18.2.2-954.g82316b04.el9 ceph 45 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:18.2.2-954.g82316b04.el9 ceph 127 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T07:52:47.805 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T07:52:47.806 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T07:52:47.807 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T07:52:47.808 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:52:47.809 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-23T07:52:47.810 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================== 2024-03-23T07:52:47.810 INFO:teuthology.orchestra.run.smithi132.stdout:Install 132 Packages 2024-03-23T07:52:47.810 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 24 Packages 2024-03-23T07:52:47.810 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:52:47.811 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 204 M 2024-03-23T07:52:47.811 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-23T07:52:48.627 INFO:teuthology.orchestra.run.smithi132.stdout:(1/156): ceph-18.2.2-954.g82316b04.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2024-03-23T07:52:48.961 INFO:teuthology.orchestra.run.smithi132.stdout:(2/156): ceph-fuse-18.2.2-954.g82316b04.el9.x86 2.5 MB/s | 843 kB 00:00 2024-03-23T07:52:49.035 INFO:teuthology.orchestra.run.smithi132.stdout:(3/156): ceph-immutable-object-cache-18.2.2-954 1.9 MB/s | 143 kB 00:00 2024-03-23T07:52:49.361 INFO:teuthology.orchestra.run.smithi132.stdout:(4/156): ceph-mds-18.2.2-954.g82316b04.el9.x86_ 6.5 MB/s | 2.1 MB 00:00 2024-03-23T07:52:49.470 INFO:teuthology.orchestra.run.smithi132.stdout:(5/156): ceph-base-18.2.2-954.g82316b04.el9.x86 4.5 MB/s | 5.1 MB 00:01 2024-03-23T07:52:49.645 INFO:teuthology.orchestra.run.smithi132.stdout:(6/156): ceph-mgr-18.2.2-954.g82316b04.el9.x86_ 5.1 MB/s | 1.4 MB 00:00 2024-03-23T07:52:50.070 INFO:teuthology.orchestra.run.smithi132.stdout:(7/156): ceph-mon-18.2.2-954.g82316b04.el9.x86_ 7.8 MB/s | 4.7 MB 00:00 2024-03-23T07:52:50.804 INFO:teuthology.orchestra.run.smithi132.stdout:(8/156): ceph-common-18.2.2-954.g82316b04.el9.x 7.4 MB/s | 18 MB 00:02 2024-03-23T07:52:50.871 INFO:teuthology.orchestra.run.smithi132.stdout:(9/156): ceph-selinux-18.2.2-954.g82316b04.el9. 384 kB/s | 25 kB 00:00 2024-03-23T07:52:51.029 INFO:teuthology.orchestra.run.smithi132.stdout:(10/156): ceph-radosgw-18.2.2-954.g82316b04.el9 8.1 MB/s | 7.7 MB 00:00 2024-03-23T07:52:51.121 INFO:teuthology.orchestra.run.smithi132.stdout:(11/156): cephfs-mirror-18.2.2-954.g82316b04.el 2.4 MB/s | 221 kB 00:00 2024-03-23T07:52:51.388 INFO:teuthology.orchestra.run.smithi132.stdout:(12/156): ceph-osd-18.2.2-954.g82316b04.el9.x86 9.5 MB/s | 16 MB 00:01 2024-03-23T07:52:51.413 INFO:teuthology.orchestra.run.smithi132.stdout:(13/156): libcephfs-devel-18.2.2-954.g82316b04. 107 kB/s | 31 kB 00:00 2024-03-23T07:52:51.480 INFO:teuthology.orchestra.run.smithi132.stdout:(14/156): libcephsqlite-18.2.2-954.g82316b04.el 2.5 MB/s | 166 kB 00:00 2024-03-23T07:52:51.530 INFO:teuthology.orchestra.run.smithi132.stdout:(15/156): libcephfs2-18.2.2-954.g82316b04.el9.x 4.9 MB/s | 704 kB 00:00 2024-03-23T07:52:51.555 INFO:teuthology.orchestra.run.smithi132.stdout:(16/156): librados-devel-18.2.2-954.g82316b04.e 1.7 MB/s | 127 kB 00:00 2024-03-23T07:52:51.614 INFO:teuthology.orchestra.run.smithi132.stdout:(17/156): libradosstriper1-18.2.2-954.g82316b04 5.6 MB/s | 474 kB 00:00 2024-03-23T07:52:51.727 INFO:teuthology.orchestra.run.smithi132.stdout:(18/156): python3-ceph-argparse-18.2.2-954.g823 400 kB/s | 45 kB 00:00 2024-03-23T07:52:51.840 INFO:teuthology.orchestra.run.smithi132.stdout:(19/156): python3-ceph-common-18.2.2-954.g82316 1.1 MB/s | 127 kB 00:00 2024-03-23T07:52:51.948 INFO:teuthology.orchestra.run.smithi132.stdout:(20/156): python3-cephfs-18.2.2-954.g82316b04.e 1.5 MB/s | 161 kB 00:00 2024-03-23T07:52:52.090 INFO:teuthology.orchestra.run.smithi132.stdout:(21/156): librgw2-18.2.2-954.g82316b04.el9.x86_ 8.4 MB/s | 4.5 MB 00:00 2024-03-23T07:52:52.132 INFO:teuthology.orchestra.run.smithi132.stdout:(22/156): python3-rados-18.2.2-954.g82316b04.el 1.7 MB/s | 320 kB 00:00 2024-03-23T07:52:52.174 INFO:teuthology.orchestra.run.smithi132.stdout:(23/156): python3-rbd-18.2.2-954.g82316b04.el9. 3.5 MB/s | 297 kB 00:00 2024-03-23T07:52:52.200 INFO:teuthology.orchestra.run.smithi132.stdout:(24/156): python3-rgw-18.2.2-954.g82316b04.el9. 1.5 MB/s | 99 kB 00:00 2024-03-23T07:52:52.241 INFO:teuthology.orchestra.run.smithi132.stdout:(25/156): rbd-fuse-18.2.2-954.g82316b04.el9.x86 1.3 MB/s | 87 kB 00:00 2024-03-23T07:52:52.334 INFO:teuthology.orchestra.run.smithi132.stdout:(26/156): rbd-nbd-18.2.2-954.g82316b04.el9.x86_ 1.9 MB/s | 172 kB 00:00 2024-03-23T07:52:52.451 INFO:teuthology.orchestra.run.smithi132.stdout:(27/156): ceph-grafana-dashboards-18.2.2-954.g8 201 kB/s | 23 kB 00:00 2024-03-23T07:52:52.576 INFO:teuthology.orchestra.run.smithi132.stdout:(28/156): rbd-mirror-18.2.2-954.g82316b04.el9.x 8.0 MB/s | 3.0 MB 00:00 2024-03-23T07:52:52.609 INFO:teuthology.orchestra.run.smithi132.stdout:(29/156): ceph-mgr-cephadm-18.2.2-954.g82316b04 876 kB/s | 138 kB 00:00 2024-03-23T07:52:52.843 INFO:teuthology.orchestra.run.smithi132.stdout:(30/156): ceph-mgr-dashboard-18.2.2-954.g82316b 6.6 MB/s | 1.7 MB 00:00 2024-03-23T07:52:52.935 INFO:teuthology.orchestra.run.smithi132.stdout:(31/156): ceph-mgr-modules-core-18.2.2-954.g823 2.6 MB/s | 245 kB 00:00 2024-03-23T07:52:53.027 INFO:teuthology.orchestra.run.smithi132.stdout:(32/156): ceph-mgr-rook-18.2.2-954.g82316b04.el 544 kB/s | 49 kB 00:00 2024-03-23T07:52:53.110 INFO:teuthology.orchestra.run.smithi132.stdout:(33/156): ceph-prometheus-alerts-18.2.2-954.g82 181 kB/s | 15 kB 00:00 2024-03-23T07:52:53.202 INFO:teuthology.orchestra.run.smithi132.stdout:(34/156): ceph-volume-18.2.2-954.g82316b04.el9. 2.8 MB/s | 264 kB 00:00 2024-03-23T07:52:53.395 INFO:teuthology.orchestra.run.smithi132.stdout:(35/156): ceph-mgr-diskprediction-local-18.2.2- 9.4 MB/s | 7.4 MB 00:00 2024-03-23T07:52:53.428 INFO:teuthology.orchestra.run.smithi132.stdout:(36/156): cephadm-18.2.2-954.g82316b04.el9.noar 998 kB/s | 224 kB 00:00 2024-03-23T07:52:53.478 INFO:teuthology.orchestra.run.smithi132.stdout:(37/156): cephfs-top-18.2.2-954.g82316b04.el9.n 221 kB/s | 18 kB 00:00 2024-03-23T07:52:54.046 INFO:teuthology.orchestra.run.smithi132.stdout:(38/156): ceph-test-18.2.2-954.g82316b04.el9.x8 13 MB/s | 40 MB 00:03 2024-03-23T07:52:54.145 INFO:teuthology.orchestra.run.smithi132.stdout:(39/156): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 920 kB 00:00 2024-03-23T07:52:54.213 INFO:teuthology.orchestra.run.smithi132.stdout:(40/156): automake-1.16.2-8.el9.noarch.rpm 845 kB/s | 662 kB 00:00 2024-03-23T07:52:54.246 INFO:teuthology.orchestra.run.smithi132.stdout:(41/156): boost-program-options-1.75.0-8.el9.x8 541 kB/s | 106 kB 00:00 2024-03-23T07:52:54.280 INFO:teuthology.orchestra.run.smithi132.stdout:(42/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 307 kB 00:00 2024-03-23T07:52:54.305 INFO:teuthology.orchestra.run.smithi132.stdout:(43/156): flexiblas-3.0.4-8.el9.x86_64.rpm 359 kB/s | 32 kB 00:00 2024-03-23T07:52:54.438 INFO:teuthology.orchestra.run.smithi132.stdout:(44/156): libaio-devel-0.3.111-13.el9.x86_64.rp 78 kB/s | 10 kB 00:00 2024-03-23T07:52:54.464 INFO:teuthology.orchestra.run.smithi132.stdout:(45/156): flexiblas-openblas-openmp-3.0.4-8.el9 111 kB/s | 17 kB 00:00 2024-03-23T07:52:54.581 INFO:teuthology.orchestra.run.smithi132.stdout:(46/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.0 MB/s | 3.0 MB 00:00 2024-03-23T07:52:54.639 INFO:teuthology.orchestra.run.smithi132.stdout:(47/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 797 kB/s | 159 kB 00:00 2024-03-23T07:52:54.665 INFO:teuthology.orchestra.run.smithi132.stdout:(48/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 229 kB/s | 44 kB 00:00 2024-03-23T07:52:54.706 INFO:teuthology.orchestra.run.smithi132.stdout:(49/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.3 MB/s | 661 kB 00:00 2024-03-23T07:52:54.740 INFO:teuthology.orchestra.run.smithi132.stdout:(50/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 677 kB/s | 46 kB 00:00 2024-03-23T07:52:54.773 INFO:teuthology.orchestra.run.smithi132.stdout:(51/156): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2024-03-23T07:52:54.807 INFO:teuthology.orchestra.run.smithi132.stdout:(52/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 282 kB 00:00 2024-03-23T07:52:54.841 INFO:teuthology.orchestra.run.smithi132.stdout:(53/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.2 MB/s | 217 kB 00:00 2024-03-23T07:52:54.858 INFO:teuthology.orchestra.run.smithi132.stdout:(54/156): openblas-0.3.21-2.el9.x86_64.rpm 419 kB/s | 34 kB 00:00 2024-03-23T07:52:55.100 INFO:teuthology.orchestra.run.smithi132.stdout:(55/156): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-03-23T07:52:55.194 INFO:teuthology.orchestra.run.smithi132.stdout:(56/156): postgresql-contrib-13.11-1.el9.x86_64 2.4 MB/s | 811 kB 00:00 2024-03-23T07:52:55.250 INFO:teuthology.orchestra.run.smithi132.stdout:(57/156): postgresql-13.11-1.el9.x86_64.rpm 4.0 MB/s | 1.5 MB 00:00 2024-03-23T07:52:55.275 INFO:teuthology.orchestra.run.smithi132.stdout:(58/156): postgresql-private-libs-13.11-1.el9.x 811 kB/s | 134 kB 00:00 2024-03-23T07:52:55.450 INFO:teuthology.orchestra.run.smithi132.stdout:(59/156): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2024-03-23T07:52:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:(60/156): python3-jsonpatch-1.21-16.el9.noarch. 131 kB/s | 24 kB 00:00 2024-03-23T07:52:55.626 INFO:teuthology.orchestra.run.smithi132.stdout:(61/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-03-23T07:52:55.651 INFO:teuthology.orchestra.run.smithi132.stdout:(62/156): python3-jsonpointer-2.0-4.el9.noarch. 112 kB/s | 18 kB 00:00 2024-03-23T07:52:55.676 INFO:teuthology.orchestra.run.smithi132.stdout:(63/156): python3-mako-1.1.4-6.el9.noarch.rpm 777 kB/s | 144 kB 00:00 2024-03-23T07:52:55.718 INFO:teuthology.orchestra.run.smithi132.stdout:(64/156): python3-markupsafe-1.1.1-12.el9.x86_6 786 kB/s | 32 kB 00:00 2024-03-23T07:52:55.743 INFO:teuthology.orchestra.run.smithi132.stdout:(65/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 217 kB 00:00 2024-03-23T07:52:55.903 INFO:teuthology.orchestra.run.smithi132.stdout:(66/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-23T07:52:55.936 INFO:teuthology.orchestra.run.smithi132.stdout:(67/156): python3-oauthlib-3.1.1-5.el9.noarch.r 806 kB/s | 174 kB 00:00 2024-03-23T07:52:55.961 INFO:teuthology.orchestra.run.smithi132.stdout:(68/156): python3-packaging-20.9-5.el9.noarch.r 338 kB/s | 69 kB 00:00 2024-03-23T07:52:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:(69/156): python3-prettytable-0.7.2-27.el9.noar 510 kB/s | 41 kB 00:00 2024-03-23T07:52:56.028 INFO:teuthology.orchestra.run.smithi132.stdout:(70/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-23T07:52:56.062 INFO:teuthology.orchestra.run.smithi132.stdout:(71/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 211 kB 00:00 2024-03-23T07:52:56.087 INFO:teuthology.orchestra.run.smithi132.stdout:(72/156): python3-pytz-2021.1-5.el9.noarch.rpm 474 kB/s | 47 kB 00:00 2024-03-23T07:52:56.112 INFO:teuthology.orchestra.run.smithi132.stdout:(73/156): python3-requests-oauthlib-1.3.0-12.el 523 kB/s | 43 kB 00:00 2024-03-23T07:52:56.155 INFO:teuthology.orchestra.run.smithi132.stdout:(74/156): python3-toml-0.10.2-6.el9.noarch.rpm 591 kB/s | 39 kB 00:00 2024-03-23T07:52:56.280 INFO:teuthology.orchestra.run.smithi132.stdout:(75/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 300 kB 00:00 2024-03-23T07:52:56.505 INFO:teuthology.orchestra.run.smithi132.stdout:(76/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-23T07:52:56.530 INFO:teuthology.orchestra.run.smithi132.stdout:(77/156): uuid-1.6.2-55.el9.x86_64.rpm 156 kB/s | 56 kB 00:00 2024-03-23T07:52:56.555 INFO:teuthology.orchestra.run.smithi132.stdout:(78/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 238 kB/s | 63 kB 00:00 2024-03-23T07:52:56.581 INFO:teuthology.orchestra.run.smithi132.stdout:(79/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 907 kB/s | 44 kB 00:00 2024-03-23T07:52:56.606 INFO:teuthology.orchestra.run.smithi132.stdout:(80/156): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-23T07:52:56.639 INFO:teuthology.orchestra.run.smithi132.stdout:(81/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-03-23T07:52:56.665 INFO:teuthology.orchestra.run.smithi132.stdout:(82/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-23T07:52:56.715 INFO:teuthology.orchestra.run.smithi132.stdout:(83/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-23T07:52:56.731 INFO:teuthology.orchestra.run.smithi132.stdout:(84/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-23T07:52:56.757 INFO:teuthology.orchestra.run.smithi132.stdout:(85/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 187 kB 00:00 2024-03-23T07:52:57.049 INFO:teuthology.orchestra.run.smithi132.stdout:(86/156): jq-1.6-16.el9.x86_64.rpm 595 kB/s | 187 kB 00:00 2024-03-23T07:52:57.074 INFO:teuthology.orchestra.run.smithi132.stdout:(87/156): libquadmath-11.4.1-3.el9.x86_64.rpm 617 kB/s | 195 kB 00:00 2024-03-23T07:52:57.125 INFO:teuthology.orchestra.run.smithi132.stdout:(88/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-03-23T07:52:57.150 INFO:teuthology.orchestra.run.smithi132.stdout:(89/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-23T07:52:57.175 INFO:teuthology.orchestra.run.smithi132.stdout:(90/156): elfutils-libelf-devel-0.190-2.el9.x86 216 kB/s | 21 kB 00:00 2024-03-23T07:52:57.201 INFO:teuthology.orchestra.run.smithi132.stdout:(91/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 337 kB/s | 25 kB 00:00 2024-03-23T07:52:57.284 INFO:teuthology.orchestra.run.smithi132.stdout:(92/156): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-23T07:52:57.318 INFO:teuthology.orchestra.run.smithi132.stdout:(93/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-03-23T07:52:57.376 INFO:teuthology.orchestra.run.smithi132.stdout:(94/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-03-23T07:52:57.485 INFO:teuthology.orchestra.run.smithi132.stdout:(95/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 12 MB/s | 4.1 MB 00:00 2024-03-23T07:52:57.610 INFO:teuthology.orchestra.run.smithi132.stdout:(96/156): fmt-8.1.1-5.el9.x86_64.rpm 379 kB/s | 111 kB 00:00 2024-03-23T07:52:57.677 INFO:teuthology.orchestra.run.smithi132.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 398 kB/s | 26 kB 00:00 2024-03-23T07:52:57.719 INFO:teuthology.orchestra.run.smithi132.stdout:(98/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 904 kB/s | 309 kB 00:00 2024-03-23T07:52:57.745 INFO:teuthology.orchestra.run.smithi132.stdout:(99/156): liboath-2.6.7-2.el9.x86_64.rpm 731 kB/s | 49 kB 00:00 2024-03-23T07:52:57.778 INFO:teuthology.orchestra.run.smithi132.stdout:(100/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-23T07:52:57.928 INFO:teuthology.orchestra.run.smithi132.stdout:(101/156): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-03-23T07:52:57.978 INFO:teuthology.orchestra.run.smithi132.stdout:(102/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-03-23T07:52:58.004 INFO:teuthology.orchestra.run.smithi132.stdout:(103/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 585 kB/s | 43 kB 00:00 2024-03-23T07:52:58.046 INFO:teuthology.orchestra.run.smithi132.stdout:(104/156): python3-cachetools-4.2.4-1.el9.noarc 486 kB/s | 32 kB 00:00 2024-03-23T07:52:58.080 INFO:teuthology.orchestra.run.smithi132.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-03-23T07:52:58.147 INFO:teuthology.orchestra.run.smithi132.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-23T07:52:58.197 INFO:teuthology.orchestra.run.smithi132.stdout:(107/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-23T07:52:58.239 INFO:teuthology.orchestra.run.smithi132.stdout:(108/156): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-23T07:52:58.264 INFO:teuthology.orchestra.run.smithi132.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-03-23T07:52:58.306 INFO:teuthology.orchestra.run.smithi132.stdout:(110/156): python3-jaraco-classes-3.2.1-5.el9.n 268 kB/s | 18 kB 00:00 2024-03-23T07:52:58.331 INFO:teuthology.orchestra.run.smithi132.stdout:(111/156): python3-jaraco-collections-3.0.0-8.e 349 kB/s | 23 kB 00:00 2024-03-23T07:52:58.423 INFO:teuthology.orchestra.run.smithi132.stdout:(112/156): libarrow-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-03-23T07:52:58.449 INFO:teuthology.orchestra.run.smithi132.stdout:(113/156): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-03-23T07:52:58.474 INFO:teuthology.orchestra.run.smithi132.stdout:(114/156): python3-jaraco-text-3.2.0-6.el9.noar 139 kB/s | 20 kB 00:00 2024-03-23T07:52:58.541 INFO:teuthology.orchestra.run.smithi132.stdout:(115/156): python3-logutils-0.3.5-21.el9.noarch 706 kB/s | 46 kB 00:00 2024-03-23T07:52:58.574 INFO:teuthology.orchestra.run.smithi132.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 794 kB/s | 79 kB 00:00 2024-03-23T07:52:58.616 INFO:teuthology.orchestra.run.smithi132.stdout:(117/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-03-23T07:52:58.642 INFO:teuthology.orchestra.run.smithi132.stdout:(118/156): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-03-23T07:52:58.708 INFO:teuthology.orchestra.run.smithi132.stdout:(119/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-23T07:52:58.734 INFO:teuthology.orchestra.run.smithi132.stdout:(120/156): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-03-23T07:52:58.871 INFO:teuthology.orchestra.run.smithi132.stdout:(121/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 394 kB/s | 90 kB 00:00 2024-03-23T07:52:58.893 INFO:teuthology.orchestra.run.smithi132.stdout:(122/156): python3-repoze-lru-0.7-16.el9.noarch 167 kB/s | 31 kB 00:00 2024-03-23T07:52:58.926 INFO:teuthology.orchestra.run.smithi132.stdout:(123/156): python3-routes-2.5.1-5.el9.noarch.rp 983 kB/s | 188 kB 00:00 2024-03-23T07:52:58.943 INFO:teuthology.orchestra.run.smithi132.stdout:(124/156): python3-rsa-4.9-2.el9.noarch.rpm 828 kB/s | 59 kB 00:00 2024-03-23T07:52:58.961 INFO:teuthology.orchestra.run.smithi132.stdout:(125/156): python3-tempora-5.0.0-2.el9.noarch.r 538 kB/s | 36 kB 00:00 2024-03-23T07:52:58.985 INFO:teuthology.orchestra.run.smithi132.stdout:(126/156): python3-typing-extensions-4.4.0-2.el 878 kB/s | 51 kB 00:00 2024-03-23T07:52:59.019 INFO:teuthology.orchestra.run.smithi132.stdout:(127/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-23T07:52:59.044 INFO:teuthology.orchestra.run.smithi132.stdout:(128/156): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-23T07:52:59.078 INFO:teuthology.orchestra.run.smithi132.stdout:(129/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-03-23T07:52:59.103 INFO:teuthology.orchestra.run.smithi132.stdout:(130/156): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-03-23T07:52:59.137 INFO:teuthology.orchestra.run.smithi132.stdout:(131/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-23T07:52:59.304 INFO:teuthology.orchestra.run.smithi132.stdout:(132/156): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-03-23T07:52:59.354 INFO:teuthology.orchestra.run.smithi132.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 6.8 MB/s | 321 kB 00:00 2024-03-23T07:52:59.554 INFO:teuthology.orchestra.run.smithi132.stdout:(134/156): librados2-18.2.2-954.g82316b04.el9.x 7.3 MB/s | 3.3 MB 00:00 2024-03-23T07:52:59.680 INFO:teuthology.orchestra.run.smithi132.stdout:(135/156): NetworkManager-1.47.2-1.el9.x86_64.r 7.1 MB/s | 2.3 MB 00:00 2024-03-23T07:52:59.763 INFO:teuthology.orchestra.run.smithi132.stdout:(136/156): librbd1-18.2.2-954.g82316b04.el9.x86 4.8 MB/s | 3.0 MB 00:00 2024-03-23T07:52:59.813 INFO:teuthology.orchestra.run.smithi132.stdout:(137/156): NetworkManager-team-1.47.2-1.el9.x86 317 kB/s | 42 kB 00:00 2024-03-23T07:52:59.889 INFO:teuthology.orchestra.run.smithi132.stdout:(138/156): NetworkManager-libnm-1.47.2-1.el9.x8 5.5 MB/s | 1.8 MB 00:00 2024-03-23T07:52:59.922 INFO:teuthology.orchestra.run.smithi132.stdout:(139/156): NetworkManager-tui-1.47.2-1.el9.x86_ 1.5 MB/s | 248 kB 00:00 2024-03-23T07:52:59.948 INFO:teuthology.orchestra.run.smithi132.stdout:(140/156): acl-2.3.1-4.el9.x86_64.rpm 536 kB/s | 71 kB 00:00 2024-03-23T07:52:59.973 INFO:teuthology.orchestra.run.smithi132.stdout:(141/156): elfutils-debuginfod-client-0.190-2.e 439 kB/s | 36 kB 00:00 2024-03-23T07:53:00.007 INFO:teuthology.orchestra.run.smithi132.stdout:(142/156): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 192 kB 00:00 2024-03-23T07:53:00.040 INFO:teuthology.orchestra.run.smithi132.stdout:(143/156): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-23T07:53:00.065 INFO:teuthology.orchestra.run.smithi132.stdout:(144/156): libacl-2.3.1-4.el9.x86_64.rpm 252 kB/s | 23 kB 00:00 2024-03-23T07:53:00.099 INFO:teuthology.orchestra.run.smithi132.stdout:(145/156): libblkid-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 108 kB 00:00 2024-03-23T07:53:00.141 INFO:teuthology.orchestra.run.smithi132.stdout:(146/156): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-23T07:53:00.175 INFO:teuthology.orchestra.run.smithi132.stdout:(147/156): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-23T07:53:00.200 INFO:teuthology.orchestra.run.smithi132.stdout:(148/156): libsmartcols-2.37.4-18.el9.x86_64.rp 638 kB/s | 64 kB 00:00 2024-03-23T07:53:00.225 INFO:teuthology.orchestra.run.smithi132.stdout:(149/156): libuuid-2.37.4-18.el9.x86_64.rpm 346 kB/s | 29 kB 00:00 2024-03-23T07:53:00.325 INFO:teuthology.orchestra.run.smithi132.stdout:(150/156): openssl-3.0.7-27.el9.x86_64.rpm 8.1 MB/s | 1.2 MB 00:00 2024-03-23T07:53:00.451 INFO:teuthology.orchestra.run.smithi132.stdout:(151/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.6 MB/s | 2.2 MB 00:00 2024-03-23T07:53:00.498 INFO:teuthology.orchestra.run.smithi132.stdout:(152/156): util-linux-core-2.37.4-18.el9.x86_64 2.6 MB/s | 465 kB 00:00 2024-03-23T07:53:00.540 INFO:teuthology.orchestra.run.smithi132.stdout:(153/156): util-linux-user-2.37.4-18.el9.x86_64 352 kB/s | 31 kB 00:00 2024-03-23T07:53:00.624 INFO:teuthology.orchestra.run.smithi132.stdout:(154/156): util-linux-2.37.4-18.el9.x86_64.rpm 5.7 MB/s | 2.3 MB 00:00 2024-03-23T07:53:00.682 INFO:teuthology.orchestra.run.smithi132.stdout:(155/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.9 MB/s | 1.1 MB 00:00 2024-03-23T07:53:00.699 INFO:teuthology.orchestra.run.smithi132.stdout:(156/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 470 kB/s | 74 kB 00:00 2024-03-23T07:53:00.713 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-23T07:53:00.714 INFO:teuthology.orchestra.run.smithi132.stdout:Total 16 MB/s | 204 MB 00:12 2024-03-23T07:53:01.972 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-23T07:53:02.340 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-23T07:53:02.340 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-23T07:53:04.487 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-23T07:53:04.487 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-23T07:53:11.175 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-23T07:53:11.290 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-23T07:53:11.330 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-23T07:53:11.402 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T07:53:11.445 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T07:53:11.541 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/180 2024-03-23T07:53:11.677 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/180 2024-03-23T07:53:11.729 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/180 2024-03-23T07:53:11.759 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-23T07:53:11.834 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-23T07:53:12.244 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-23T07:53:12.290 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T07:53:12.420 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T07:53:12.449 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T07:53:12.603 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T07:53:12.643 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T07:53:12.710 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-23T07:53:12.997 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-23T07:53:13.055 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-23T07:53:13.111 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T07:53:13.261 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T07:53:13.299 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T07:53:13.389 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T07:53:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-23T07:53:13.482 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-23T07:53:13.801 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-23T07:53:13.957 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T07:53:14.045 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T07:53:14.234 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T07:53:14.318 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T07:53:14.747 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T07:53:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-23T07:53:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-23T07:53:14.902 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-23T07:53:15.019 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-23T07:53:15.081 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-23T07:53:15.131 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-23T07:53:15.212 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-23T07:53:15.253 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-23T07:53:15.287 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-23T07:53:15.393 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-23T07:53:15.434 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 33/180 2024-03-23T07:53:15.487 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 34/180 2024-03-23T07:53:15.600 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 35/180 2024-03-23T07:53:15.769 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-23T07:53:15.836 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-23T07:53:16.521 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-23T07:53:16.641 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-23T07:53:16.985 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-23T07:53:17.035 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T07:53:17.069 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T07:53:17.140 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-23T07:53:17.192 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 43/180 2024-03-23T07:53:17.222 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-23T07:53:17.622 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-23T07:53:17.872 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-23T07:53:17.981 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-23T07:53:18.042 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-23T07:53:18.251 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-23T07:53:18.299 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-23T07:53:18.481 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-23T07:53:18.519 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-23T07:53:18.585 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-23T07:53:18.679 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-23T07:53:18.720 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-23T07:53:18.758 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-23T07:53:18.823 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-23T07:53:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-23T07:53:18.975 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-23T07:53:19.171 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-23T07:53:19.218 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-23T07:53:19.253 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-23T07:53:19.379 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-23T07:53:19.417 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 64/180 2024-03-23T07:53:19.461 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-23T07:53:19.520 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-23T07:53:19.563 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-23T07:53:19.719 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-23T07:53:19.775 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-23T07:53:19.853 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-23T07:53:19.935 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-23T07:53:20.186 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-23T07:53:20.253 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-23T07:53:20.942 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-23T07:53:20.993 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/180 2024-03-23T07:53:21.031 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-23T07:53:21.070 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-23T07:53:21.102 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-23T07:53:21.147 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-23T07:53:21.207 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-23T07:53:21.247 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-23T07:53:21.375 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-23T07:53:21.418 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-23T07:53:21.454 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-23T07:53:21.517 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-23T07:53:21.556 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-23T07:53:21.698 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-23T07:53:21.746 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-23T07:53:21.808 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-23T07:53:21.953 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-23T07:53:22.019 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-23T07:53:22.981 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-23T07:53:23.026 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-23T07:53:23.060 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-23T07:53:23.117 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-23T07:53:23.367 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-23T07:53:23.397 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-23T07:53:23.504 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-23T07:53:24.289 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-23T07:53:24.419 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-23T07:53:25.929 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-23T07:53:26.030 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-23T07:53:26.068 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-23T07:53:26.099 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-23T07:53:26.140 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-23T07:53:26.506 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-23T07:53:26.570 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T07:53:26.686 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T07:53:26.718 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 108/180 2024-03-23T07:53:26.783 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-23T07:53:27.272 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-23T07:53:27.307 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T07:53:27.399 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T07:53:27.426 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 112/180 2024-03-23T07:53:29.871 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T07:53:29.907 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T07:53:30.516 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T07:53:30.582 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T07:53:30.736 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T07:53:30.736 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T07:53:30.736 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T07:53:30.736 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:30.778 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T07:53:43.457 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T07:53:43.457 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-03-23T07:53:43.458 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-03-23T07:53:43.458 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-03-23T07:53:43.458 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-03-23T07:53:43.458 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-03-23T07:53:43.458 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-03-23T07:53:43.458 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-03-23T07:53:43.458 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:43.667 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T07:53:43.717 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T07:53:43.717 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:43.717 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T07:53:43.717 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:45.310 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T07:53:45.387 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T07:53:45.388 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:45.388 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T07:53:45.388 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T07:53:45.388 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T07:53:45.388 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:45.611 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T07:53:45.678 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T07:53:45.678 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:45.678 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T07:53:45.678 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T07:53:45.678 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T07:53:45.678 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:46.153 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T07:53:46.224 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T07:53:46.224 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:46.224 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T07:53:46.224 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T07:53:46.225 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T07:53:46.225 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:46.347 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-23T07:53:46.995 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T07:53:47.040 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T07:53:47.087 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 122/180 2024-03-23T07:53:47.201 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 123/180 2024-03-23T07:53:47.235 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T07:53:47.541 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T07:53:47.603 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T07:53:48.741 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T07:53:48.778 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T07:53:48.920 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T07:53:49.092 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 127/180 2024-03-23T07:53:49.130 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T07:53:49.205 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T07:53:49.205 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:49.205 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T07:53:49.205 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T07:53:49.205 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T07:53:49.206 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:49.264 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T07:53:49.303 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T07:53:49.408 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:18.2.2-954.g82316b04.el9.x86_64 130/180 2024-03-23T07:53:50.240 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-23T07:53:50.275 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T07:53:50.342 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T07:53:50.343 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:50.343 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T07:53:50.343 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T07:53:50.343 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T07:53:50.343 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:50.392 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T07:53:50.460 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T07:53:50.460 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:50.461 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T07:53:50.461 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:50.535 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T07:53:50.609 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T07:53:50.609 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:50.609 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T07:53:50.609 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T07:53:50.609 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T07:53:50.609 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:50.960 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T07:53:51.031 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T07:53:51.032 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:53:51.032 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T07:53:51.032 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T07:53:51.032 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T07:53:51.032 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:53:53.725 INFO:teuthology.orchestra.run.smithi105.stdout:(86/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.4 kB/s | 195 kB 01:20 2024-03-23T07:53:53.757 INFO:teuthology.orchestra.run.smithi105.stdout:(87/156): jq-1.6-16.el9.x86_64.rpm 2.3 kB/s | 187 kB 01:20 2024-03-23T07:53:53.841 INFO:teuthology.orchestra.run.smithi105.stdout:(88/156): libgfortran-11.4.1-3.el9.x86_64.rpm 9.9 kB/s | 803 kB 01:20 2024-03-23T07:53:53.958 INFO:teuthology.orchestra.run.smithi105.stdout:(89/156): elfutils-libelf-devel-0.190-2.el9.x86 108 kB/s | 21 kB 00:00 2024-03-23T07:53:53.983 INFO:teuthology.orchestra.run.smithi105.stdout:(90/156): python3-requests-2.25.1-8.el9.noarch. 495 kB/s | 125 kB 00:00 2024-03-23T07:53:54.009 INFO:teuthology.orchestra.run.smithi105.stdout:(91/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 153 kB/s | 25 kB 00:00 2024-03-23T07:53:54.167 INFO:teuthology.orchestra.run.smithi105.stdout:(92/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-03-23T07:53:54.251 INFO:teuthology.orchestra.run.smithi105.stdout:(93/156): python3-devel-3.9.17-2.el9.x86_64.rpm 920 kB/s | 245 kB 00:00 2024-03-23T07:53:54.276 INFO:teuthology.orchestra.run.smithi105.stdout:(94/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 868 kB/s | 94 kB 00:00 2024-03-23T07:53:54.410 INFO:teuthology.orchestra.run.smithi105.stdout:(95/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 9.2 MB/s | 4.1 MB 00:00 2024-03-23T07:53:54.527 INFO:teuthology.orchestra.run.smithi105.stdout:(96/156): fmt-8.1.1-5.el9.x86_64.rpm 403 kB/s | 111 kB 00:00 2024-03-23T07:53:54.594 INFO:teuthology.orchestra.run.smithi105.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 395 kB/s | 26 kB 00:00 2024-03-23T07:53:54.619 INFO:teuthology.orchestra.run.smithi105.stdout:(98/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 904 kB/s | 309 kB 00:00 2024-03-23T07:53:54.661 INFO:teuthology.orchestra.run.smithi105.stdout:(99/156): liboath-2.6.7-2.el9.x86_64.rpm 734 kB/s | 49 kB 00:00 2024-03-23T07:53:54.686 INFO:teuthology.orchestra.run.smithi105.stdout:(100/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-23T07:53:54.812 INFO:teuthology.orchestra.run.smithi105.stdout:(101/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-03-23T07:53:54.904 INFO:teuthology.orchestra.run.smithi105.stdout:(102/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-03-23T07:53:54.929 INFO:teuthology.orchestra.run.smithi105.stdout:(103/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 374 kB/s | 43 kB 00:00 2024-03-23T07:53:54.970 INFO:teuthology.orchestra.run.smithi105.stdout:(104/156): python3-cachetools-4.2.4-1.el9.noarc 487 kB/s | 32 kB 00:00 2024-03-23T07:53:54.996 INFO:teuthology.orchestra.run.smithi105.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 226 kB/s | 15 kB 00:00 2024-03-23T07:53:55.033 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 136/180 2024-03-23T07:53:55.037 INFO:teuthology.orchestra.run.smithi105.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-23T07:53:55.071 INFO:teuthology.orchestra.run.smithi105.stdout:(107/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-03-23T07:53:55.114 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 137/180 2024-03-23T07:53:55.157 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 138/180 2024-03-23T07:53:55.163 INFO:teuthology.orchestra.run.smithi105.stdout:(108/156): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-03-23T07:53:55.189 INFO:teuthology.orchestra.run.smithi105.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-23T07:53:55.222 INFO:teuthology.orchestra.run.smithi105.stdout:(110/156): python3-google-auth-2.28.2-1.el9.noa 1.1 MB/s | 203 kB 00:00 2024-03-23T07:53:55.247 INFO:teuthology.orchestra.run.smithi105.stdout:(111/156): python3-jaraco-classes-3.2.1-5.el9.n 213 kB/s | 18 kB 00:00 2024-03-23T07:53:55.273 INFO:teuthology.orchestra.run.smithi105.stdout:(112/156): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-03-23T07:53:55.298 INFO:teuthology.orchestra.run.smithi105.stdout:(113/156): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-03-23T07:53:55.323 INFO:teuthology.orchestra.run.smithi105.stdout:(114/156): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-03-23T07:53:55.365 INFO:teuthology.orchestra.run.smithi105.stdout:(115/156): python3-logutils-0.3.5-21.el9.noarch 701 kB/s | 46 kB 00:00 2024-03-23T07:53:55.390 INFO:teuthology.orchestra.run.smithi105.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-23T07:53:55.432 INFO:teuthology.orchestra.run.smithi105.stdout:(117/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-03-23T07:53:55.438 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-23T07:53:55.449 INFO:teuthology.orchestra.run.smithi105.stdout:(118/156): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-23T07:53:55.491 INFO:teuthology.orchestra.run.smithi105.stdout:(119/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-03-23T07:53:55.508 INFO:teuthology.orchestra.run.smithi105.stdout:(120/156): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-03-23T07:53:55.542 INFO:teuthology.orchestra.run.smithi105.stdout:(121/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 983 kB/s | 90 kB 00:00 2024-03-23T07:53:55.559 INFO:teuthology.orchestra.run.smithi105.stdout:(122/156): python3-repoze-lru-0.7-16.el9.noarch 462 kB/s | 31 kB 00:00 2024-03-23T07:53:55.562 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-23T07:53:55.584 INFO:teuthology.orchestra.run.smithi105.stdout:(123/156): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-23T07:53:55.609 INFO:teuthology.orchestra.run.smithi105.stdout:(124/156): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-03-23T07:53:55.615 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 141/180 2024-03-23T07:53:55.635 INFO:teuthology.orchestra.run.smithi105.stdout:(125/156): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-03-23T07:53:55.660 INFO:teuthology.orchestra.run.smithi105.stdout:(126/156): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-03-23T07:53:55.693 INFO:teuthology.orchestra.run.smithi105.stdout:(127/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-23T07:53:55.700 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-23T07:53:55.719 INFO:teuthology.orchestra.run.smithi105.stdout:(128/156): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-23T07:53:55.753 INFO:teuthology.orchestra.run.smithi105.stdout:(129/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-03-23T07:53:55.774 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-23T07:53:55.778 INFO:teuthology.orchestra.run.smithi105.stdout:(130/156): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-03-23T07:53:55.803 INFO:teuthology.orchestra.run.smithi105.stdout:(131/156): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-03-23T07:53:55.820 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-23T07:53:55.862 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 145/180 2024-03-23T07:53:55.907 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-23T07:53:55.958 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 147/180 2024-03-23T07:53:56.003 INFO:teuthology.orchestra.run.smithi105.stdout:(132/156): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-23T07:53:56.045 INFO:teuthology.orchestra.run.smithi105.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 7.9 MB/s | 321 kB 00:00 2024-03-23T07:53:56.056 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 148/180 2024-03-23T07:53:56.168 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-23T07:53:56.215 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-23T07:53:56.279 INFO:teuthology.orchestra.run.smithi105.stdout:(134/156): librados2-18.2.2-954.g82316b04.el9.x 6.6 MB/s | 3.3 MB 00:00 2024-03-23T07:53:56.454 INFO:teuthology.orchestra.run.smithi105.stdout:(135/156): NetworkManager-1.47.2-1.el9.x86_64.r 5.7 MB/s | 2.3 MB 00:00 2024-03-23T07:53:56.554 INFO:teuthology.orchestra.run.smithi105.stdout:(136/156): NetworkManager-libnm-1.47.2-1.el9.x8 6.7 MB/s | 1.8 MB 00:00 2024-03-23T07:53:56.580 INFO:teuthology.orchestra.run.smithi105.stdout:(137/156): NetworkManager-team-1.47.2-1.el9.x86 337 kB/s | 42 kB 00:00 2024-03-23T07:53:56.655 INFO:teuthology.orchestra.run.smithi105.stdout:(138/156): NetworkManager-tui-1.47.2-1.el9.x86_ 2.4 MB/s | 248 kB 00:00 2024-03-23T07:53:56.680 INFO:teuthology.orchestra.run.smithi105.stdout:(139/156): acl-2.3.1-4.el9.x86_64.rpm 718 kB/s | 71 kB 00:00 2024-03-23T07:53:56.764 INFO:teuthology.orchestra.run.smithi105.stdout:(140/156): elfutils-debuginfod-client-0.190-2.e 338 kB/s | 36 kB 00:00 2024-03-23T07:53:56.814 INFO:teuthology.orchestra.run.smithi105.stdout:(141/156): elfutils-libelf-0.190-2.el9.x86_64.r 1.4 MB/s | 192 kB 00:00 2024-03-23T07:53:56.881 INFO:teuthology.orchestra.run.smithi105.stdout:(142/156): libacl-2.3.1-4.el9.x86_64.rpm 350 kB/s | 23 kB 00:00 2024-03-23T07:53:56.931 INFO:teuthology.orchestra.run.smithi105.stdout:(143/156): elfutils-libs-0.190-2.el9.x86_64.rpm 1.5 MB/s | 256 kB 00:00 2024-03-23T07:53:57.007 INFO:teuthology.orchestra.run.smithi105.stdout:(144/156): librbd1-18.2.2-954.g82316b04.el9.x86 2.6 MB/s | 3.0 MB 00:01 2024-03-23T07:53:57.089 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T07:53:57.098 INFO:teuthology.orchestra.run.smithi105.stdout:(145/156): libblkid-2.37.4-18.el9.x86_64.rpm 501 kB/s | 108 kB 00:00 2024-03-23T07:53:57.124 INFO:teuthology.orchestra.run.smithi105.stdout:(146/156): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-23T07:53:57.138 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T07:53:57.149 INFO:teuthology.orchestra.run.smithi105.stdout:(147/156): libfdisk-2.37.4-18.el9.x86_64.rpm 715 kB/s | 155 kB 00:00 2024-03-23T07:53:57.201 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T07:53:57.207 INFO:teuthology.orchestra.run.smithi105.stdout:(148/156): libsmartcols-2.37.4-18.el9.x86_64.rp 592 kB/s | 64 kB 00:00 2024-03-23T07:53:57.233 INFO:teuthology.orchestra.run.smithi105.stdout:(149/156): libuuid-2.37.4-18.el9.x86_64.rpm 266 kB/s | 29 kB 00:00 2024-03-23T07:53:57.391 INFO:teuthology.orchestra.run.smithi105.stdout:(150/156): openssl-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 1.2 MB 00:00 2024-03-23T07:53:57.541 INFO:teuthology.orchestra.run.smithi105.stdout:(151/156): util-linux-core-2.37.4-18.el9.x86_64 3.1 MB/s | 465 kB 00:00 2024-03-23T07:53:57.609 INFO:teuthology.orchestra.run.smithi105.stdout:(152/156): util-linux-user-2.37.4-18.el9.x86_64 472 kB/s | 31 kB 00:00 2024-03-23T07:53:57.684 INFO:teuthology.orchestra.run.smithi105.stdout:(153/156): util-linux-2.37.4-18.el9.x86_64.rpm 5.1 MB/s | 2.3 MB 00:00 2024-03-23T07:53:57.809 INFO:teuthology.orchestra.run.smithi105.stdout:(154/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 2.2 MB 00:00 2024-03-23T07:53:57.951 INFO:teuthology.orchestra.run.smithi105.stdout:(155/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.1 MB/s | 1.1 MB 00:00 2024-03-23T07:53:58.385 INFO:teuthology.orchestra.run.smithi105.stdout:(156/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 106 kB/s | 74 kB 00:00 2024-03-23T07:53:58.393 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-23T07:53:58.393 INFO:teuthology.orchestra.run.smithi105.stdout:Total 2.2 MB/s | 204 MB 01:34 2024-03-23T07:53:58.471 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 152/180 2024-03-23T07:53:58.595 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-23T07:53:58.644 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-23T07:53:58.779 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-23T07:53:58.815 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-23T07:53:58.848 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-23T07:53:58.882 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-23T07:53:58.932 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-23T07:53:58.933 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T07:53:59.087 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T07:53:59.144 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-23T07:53:59.145 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T07:53:59.275 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T07:53:59.331 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-23T07:53:59.371 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-23T07:53:59.406 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-23T07:53:59.439 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-23T07:53:59.516 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-23T07:53:59.596 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-23T07:53:59.631 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-23T07:53:59.652 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-23T07:53:59.710 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-23T07:53:59.752 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-23T07:53:59.786 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-23T07:53:59.819 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-23T07:53:59.855 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T07:53:59.855 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T07:53:59.999 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-23T07:53:59.999 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-23T07:54:00.153 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T07:54:00.153 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T07:54:00.309 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T07:54:00.347 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-23T07:54:00.381 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-23T07:54:00.417 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-23T07:54:00.457 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-23T07:54:00.578 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T07:54:00.828 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T07:54:00.828 INFO:teuthology.orchestra.run.smithi132.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-23T07:54:00.828 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:54:02.526 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-23T07:54:02.527 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-23T07:54:05.320 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:18.2.2-954.g82316b04.el9.x86_64 1/180 2024-03-23T07:54:05.320 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2/180 2024-03-23T07:54:05.320 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 3/180 2024-03-23T07:54:05.320 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 4/180 2024-03-23T07:54:05.320 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-954.g82316b 5/180 2024-03-23T07:54:05.320 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 6/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 7/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 8/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 9/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 13/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 14/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 15/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 17/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 18/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 19/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 20/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 21/180 2024-03-23T07:54:05.321 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 22/180 2024-03-23T07:54:05.322 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 23/180 2024-03-23T07:54:05.322 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 24/180 2024-03-23T07:54:05.322 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 25/180 2024-03-23T07:54:05.322 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 26/180 2024-03-23T07:54:05.322 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 27/180 2024-03-23T07:54:05.322 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 28/180 2024-03-23T07:54:05.335 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 29/180 2024-03-23T07:54:05.335 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 30/180 2024-03-23T07:54:05.335 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 31/180 2024-03-23T07:54:05.335 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 32/180 2024-03-23T07:54:05.335 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 33/180 2024-03-23T07:54:05.335 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 34/180 2024-03-23T07:54:05.335 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 35/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 36/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:18.2.2-954.g82316b04.el9.noarch 37/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 38/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-23T07:54:05.336 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-23T07:54:05.337 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-23T07:54:05.338 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-23T07:54:05.339 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 103/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-23T07:54:05.340 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 109/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-23T07:54:05.341 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:18.2.2-954.g82316b04.el9.x86_64 133/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-23T07:54:05.342 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 139/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 141/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 143/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 145/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-23T07:54:05.343 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-23T07:54:05.344 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-23T07:54:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-23T07:54:07.656 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-23T07:54:07.656 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.657 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-23T07:54:07.658 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.659 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T07:54:07.660 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-16.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.661 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T07:54:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.663 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T07:54:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T07:54:07.665 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.666 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T07:54:07.667 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T07:54:07.668 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T07:54:07.668 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T07:54:07.668 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T07:54:07.668 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T07:54:07.668 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T07:54:07.668 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:54:07.668 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-23T07:54:08.267 DEBUG:teuthology.parallel:result is None 2024-03-23T07:54:09.017 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-23T07:54:09.157 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-23T07:54:09.195 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-23T07:54:09.233 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T07:54:09.276 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-23T07:54:09.355 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/180 2024-03-23T07:54:09.453 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/180 2024-03-23T07:54:09.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/180 2024-03-23T07:54:09.540 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-23T07:54:09.624 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-23T07:54:09.993 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-23T07:54:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T07:54:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T07:54:10.264 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T07:54:10.387 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T07:54:10.425 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T07:54:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-23T07:54:10.785 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-23T07:54:10.850 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-23T07:54:10.877 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T07:54:11.010 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T07:54:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T07:54:11.119 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 17/180 2024-03-23T07:54:11.158 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-23T07:54:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-23T07:54:11.573 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-23T07:54:11.719 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T07:54:11.807 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 21/180 2024-03-23T07:54:12.006 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T07:54:12.055 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T07:54:12.467 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 22/180 2024-03-23T07:54:12.518 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-23T07:54:12.556 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-23T07:54:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-23T07:54:12.747 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-23T07:54:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-23T07:54:12.876 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-23T07:54:12.968 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-23T07:54:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-23T07:54:13.050 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-23T07:54:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-23T07:54:13.214 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 33/180 2024-03-23T07:54:13.265 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 34/180 2024-03-23T07:54:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 35/180 2024-03-23T07:54:13.538 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-23T07:54:13.599 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-23T07:54:14.286 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-23T07:54:14.382 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-23T07:54:14.486 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-23T07:54:14.557 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T07:54:14.590 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-23T07:54:14.661 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-23T07:54:14.715 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 43/180 2024-03-23T07:54:14.756 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-23T07:54:15.168 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-23T07:54:15.357 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-23T07:54:15.460 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-23T07:54:15.516 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-23T07:54:15.714 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-23T07:54:15.779 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-23T07:54:15.920 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-23T07:54:15.966 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-23T07:54:16.031 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-23T07:54:16.117 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-23T07:54:16.152 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-23T07:54:16.188 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-23T07:54:16.254 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-23T07:54:16.313 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-23T07:54:16.375 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-23T07:54:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-23T07:54:16.607 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-23T07:54:16.652 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-23T07:54:16.810 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-23T07:54:16.864 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 64/180 2024-03-23T07:54:16.911 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-23T07:54:16.947 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-23T07:54:16.986 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-23T07:54:17.151 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-23T07:54:17.204 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-23T07:54:17.272 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-23T07:54:17.354 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-23T07:54:17.606 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-23T07:54:17.665 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-23T07:54:17.839 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-23T07:54:17.888 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 75/180 2024-03-23T07:54:17.935 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-23T07:54:17.966 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-23T07:54:18.005 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-23T07:54:18.051 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-23T07:54:18.105 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-23T07:54:18.152 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-23T07:54:18.280 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-23T07:54:18.332 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-23T07:54:18.367 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-23T07:54:18.430 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-23T07:54:18.469 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-23T07:54:18.622 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-23T07:54:18.667 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-23T07:54:18.727 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-23T07:54:18.865 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-23T07:54:18.933 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-23T07:54:19.911 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-23T07:54:19.984 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-23T07:54:20.024 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-23T07:54:20.057 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-23T07:54:20.305 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-23T07:54:20.336 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-23T07:54:20.427 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-23T07:54:21.217 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-23T07:54:21.346 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-23T07:54:22.846 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-23T07:54:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-23T07:54:22.963 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-23T07:54:22.994 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-23T07:54:23.034 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-23T07:54:23.386 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-23T07:54:23.416 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T07:54:23.535 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 107/180 2024-03-23T07:54:23.563 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 108/180 2024-03-23T07:54:23.617 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-23T07:54:24.089 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-23T07:54:24.119 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T07:54:24.211 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 111/180 2024-03-23T07:54:24.238 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 112/180 2024-03-23T07:54:26.674 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T07:54:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T07:54:27.312 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 113/180 2024-03-23T07:54:27.355 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T07:54:27.520 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 114/180 2024-03-23T07:54:27.520 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T07:54:27.520 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T07:54:27.521 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:27.572 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T07:54:40.029 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 115/180 2024-03-23T07:54:40.029 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-03-23T07:54:40.029 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-03-23T07:54:40.029 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-03-23T07:54:40.029 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-03-23T07:54:40.030 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-03-23T07:54:40.030 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-03-23T07:54:40.030 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-03-23T07:54:40.030 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:40.244 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T07:54:40.286 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 116/180 2024-03-23T07:54:40.286 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:40.286 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T07:54:40.286 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:41.804 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T07:54:41.903 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 117/180 2024-03-23T07:54:41.904 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:41.904 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T07:54:41.904 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T07:54:41.904 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T07:54:41.904 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:42.121 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T07:54:42.203 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 118/180 2024-03-23T07:54:42.204 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:42.204 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T07:54:42.204 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T07:54:42.204 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T07:54:42.204 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:42.703 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T07:54:42.798 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 119/180 2024-03-23T07:54:42.798 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:42.798 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T07:54:42.798 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T07:54:42.798 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T07:54:42.798 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:42.915 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-23T07:54:43.428 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T07:54:43.460 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:18.2.2-954.g82316b04.el9.noarch 121/180 2024-03-23T07:54:43.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 122/180 2024-03-23T07:54:43.621 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 123/180 2024-03-23T07:54:43.663 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T07:54:43.985 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 124/180 2024-03-23T07:54:44.021 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T07:54:45.139 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 125/180 2024-03-23T07:54:45.172 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T07:54:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 126/180 2024-03-23T07:54:45.477 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 127/180 2024-03-23T07:54:45.507 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T07:54:45.594 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 128/180 2024-03-23T07:54:45.594 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:45.594 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T07:54:45.595 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T07:54:45.595 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T07:54:45.595 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T07:54:45.685 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 129/180 2024-03-23T07:54:45.736 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:18.2.2-954.g82316b04.el9.x86_64 130/180 2024-03-23T07:54:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-23T07:54:46.527 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T07:54:46.612 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 132/180 2024-03-23T07:54:46.613 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:46.613 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T07:54:46.613 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T07:54:46.613 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T07:54:46.613 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:46.660 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T07:54:46.754 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-954.g82316b 133/180 2024-03-23T07:54:46.754 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:46.754 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T07:54:46.754 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:46.837 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T07:54:46.937 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 134/180 2024-03-23T07:54:46.937 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:46.937 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T07:54:46.937 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T07:54:46.938 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T07:54:46.938 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:47.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T07:54:47.379 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T07:54:47.380 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T07:54:47.380 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T07:54:47.380 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T07:54:47.380 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T07:54:47.380 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:54:51.327 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 136/180 2024-03-23T07:54:51.381 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 137/180 2024-03-23T07:54:51.416 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 138/180 2024-03-23T07:54:51.732 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-23T07:54:51.856 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-23T07:54:51.899 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 141/180 2024-03-23T07:54:51.949 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-23T07:54:52.024 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-23T07:54:52.078 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-23T07:54:52.113 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 145/180 2024-03-23T07:54:52.149 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-23T07:54:52.201 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 147/180 2024-03-23T07:54:52.290 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 148/180 2024-03-23T07:54:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-23T07:54:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-23T07:54:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T07:54:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T07:54:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-23T07:54:54.786 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 152/180 2024-03-23T07:54:54.870 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-23T07:54:54.915 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-23T07:54:55.043 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-23T07:54:55.091 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-23T07:54:55.123 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-23T07:54:55.158 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-23T07:54:55.201 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-23T07:54:55.201 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T07:54:55.345 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-23T07:54:55.387 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-23T07:54:55.387 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T07:54:55.525 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-23T07:54:55.580 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-23T07:54:55.632 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-23T07:54:55.665 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-23T07:54:55.698 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-23T07:54:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-23T07:54:55.848 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-23T07:54:55.911 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-23T07:54:55.944 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-23T07:54:55.978 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-23T07:54:56.012 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-23T07:54:56.045 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-23T07:54:56.080 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T07:54:56.081 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T07:54:56.411 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-23T07:54:56.412 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T07:54:56.551 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-23T07:54:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-23T07:54:56.615 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-23T07:54:56.651 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-23T07:54:56.691 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-23T07:54:56.810 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T07:54:57.062 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-23T07:54:57.062 INFO:teuthology.orchestra.run.smithi105.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-23T07:54:57.062 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:01.581 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:18.2.2-954.g82316b04.el9.x86_64 1/180 2024-03-23T07:55:01.581 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 3/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 4/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-954.g82316b 5/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 6/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 7/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 8/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 9/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 10/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 11/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 12/180 2024-03-23T07:55:01.582 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 13/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_6 14/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 15/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 16/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 17/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_ 18/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:18.2.2-954.g82316b04.el9.x86_64 19/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:18.2.2-954.g82316b04.el9 20/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:18.2.2-954.g82316b04.el9.x 21/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 22/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 23/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 24/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 25/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 26/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 27/180 2024-03-23T07:55:01.583 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 28/180 2024-03-23T07:55:01.601 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 29/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noar 30/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.no 31/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 32/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9 33/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 34/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 35/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 36/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:18.2.2-954.g82316b04.el9.noarch 37/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 38/180 2024-03-23T07:55:01.602 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-23T07:55:01.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-23T07:55:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-23T07:55:01.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-23T07:55:01.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 103/180 2024-03-23T07:55:01.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 109/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-23T07:55:01.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-23T07:55:01.609 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:18.2.2-954.g82316b04.el9.x86_64 133/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:18.2.2-954.g82316b04.el9.x86_64 135/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 139/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 141/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 143/180 2024-03-23T07:55:01.610 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 145/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-23T07:55:01.611 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-23T07:55:01.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-23T07:55:01.613 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-23T07:55:03.881 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T07:55:03.882 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.883 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.884 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.885 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:18.2.2-954.g82316b04.el9.noarch 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T07:55:03.886 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-16.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.887 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T07:55:03.888 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T07:55:03.889 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T07:55:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T07:55:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T07:55:03.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T07:55:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T07:55:03.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:18.2.2-954.g82316b04.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T07:55:03.895 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T07:55:03.896 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T07:55:03.896 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:03.896 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-23T07:55:04.640 DEBUG:teuthology.parallel:result is None 2024-03-23T07:55:04.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T07:55:04.788 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T07:55:04.844 INFO:teuthology.orchestra.run.smithi105.stdout:18.2.2-954.g82316b04.el9 2024-03-23T07:55:04.844 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-954.g82316b04.el9 2024-03-23T07:55:04.844 INFO:teuthology.task.install:The correct ceph version 18.2.2-954.g82316b04 is installed. 2024-03-23T07:55:04.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T07:55:04.987 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T07:55:05.058 INFO:teuthology.orchestra.run.smithi132.stdout:18.2.2-954.g82316b04.el9 2024-03-23T07:55:05.058 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-954.g82316b04.el9 2024-03-23T07:55:05.058 INFO:teuthology.task.install:The correct ceph version 18.2.2-954.g82316b04 is installed. 2024-03-23T07:55:05.060 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T07:55:05.061 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:05.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T07:55:05.137 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:05.137 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T07:55:05.240 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T07:55:05.259 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:05.259 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T07:55:05.331 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T07:55:05.445 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:05.445 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T07:55:05.546 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T07:55:05.677 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T07:55:05.677 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:05.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T07:55:05.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T07:55:05.896 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:05.897 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T07:55:05.985 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T07:55:06.108 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T07:55:06.108 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:06.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T07:55:06.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T07:55:06.303 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:06.303 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T07:55:06.389 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T07:55:06.516 INFO:teuthology.run_tasks:Running task ceph... 2024-03-23T07:55:06.583 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-23T07:55:06.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2024-03-23T07:55:06.589 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /var/log/ceph 2024-03-23T07:55:06.663 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-23T07:55:06.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-23T07:55:06.668 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-23T07:55:06.744 INFO:tasks.ceph:Creating extra log directories... 2024-03-23T07:55:06.744 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-23T07:55:06.748 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-23T07:55:06.835 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-23T07:55:06.836 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-23T07:55:06.836 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-23T07:55:06.836 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:06.836 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-23T07:55:06.876 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-23T07:55:07.085 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-23T07:55:07.158 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:07.158 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-23T07:55:07.209 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-23T07:55:07.471 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-23T07:55:07.573 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-23T07:55:07.574 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds.a'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-23T07:55:07.574 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362', 'branch': 'wip-yuri11-testing-2024-03-21-0851-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/cap-flush}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7617362', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri11-testing-2024-03-21-0851-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds.a'], 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri11-testing-2024-03-21-0851-reef', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1919, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri11-testing-2024-03-21-0851-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCvGzdXZnCG7efrJf2Bl6Dzu60db9fLBm1CeDGCfKRjVrXXEK7w2OVUopAP2ZAodOEUU7d+cxO2ZO98vccUPrtE=', 'smithi132.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjJfB8Bx/I2rR6IbWMR3RU/LMDkgYFJUwA6VV18wAW1BPYbLlpVCjOp/Htuc5OR++0aJno1/ErdH334QNCUea0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds.a'], 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cap_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-22_13:15:24', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-23T07:55:07.574 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-23T07:55:07.579 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-23T07:55:07.632 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2024-03-23T07:55:07.662 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/run/ceph 2024-03-23T07:55:07.764 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:07.764 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T07:55:07.806 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T07:55:07.806 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-03-23T07:55:07.887 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T07:55:07.887 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-23T07:55:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-23 07:54:57.533048064 +0000 2024-03-23T07:55:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-23 07:51:16.581745552 +0000 2024-03-23T07:55:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-23 07:51:16.581745552 +0000 2024-03-23T07:55:07.888 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-23 07:51:16.581745552 +0000 2024-03-23T07:55:07.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T07:55:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-23T07:55:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-23T07:55:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000365546 s, 1.4 MB/s 2024-03-23T07:55:07.999 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T07:55:08.043 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-23 07:54:57.535048024 +0000 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-23 07:51:17.126734677 +0000 2024-03-23T07:55:08.123 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-23 07:51:17.126734677 +0000 2024-03-23T07:55:08.124 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-23 07:51:17.126734677 +0000 2024-03-23T07:55:08.124 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T07:55:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-23T07:55:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-23T07:55:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000315366 s, 1.6 MB/s 2024-03-23T07:55:08.237 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T07:55:08.287 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-23 07:54:57.535048024 +0000 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-23 07:51:17.692723382 +0000 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-23 07:51:17.692723382 +0000 2024-03-23T07:55:08.330 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-23 07:51:17.692723382 +0000 2024-03-23T07:55:08.331 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T07:55:08.441 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-23T07:55:08.441 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-23T07:55:08.441 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000290859 s, 1.8 MB/s 2024-03-23T07:55:08.444 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T07:55:08.484 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-23 07:54:57.536048004 +0000 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-23 07:51:18.254712168 +0000 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-23 07:51:18.254712168 +0000 2024-03-23T07:55:08.564 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-23 07:51:18.254712168 +0000 2024-03-23T07:55:08.564 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T07:55:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-23T07:55:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-23T07:55:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.00032421 s, 1.6 MB/s 2024-03-23T07:55:08.678 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T07:55:08.719 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:08.720 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:08.720 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T07:55:08.773 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T07:55:08.773 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-23 07:54:01.244346466 +0000 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-23 07:51:25.143107086 +0000 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-23 07:51:25.143107086 +0000 2024-03-23T07:55:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-23 07:51:25.143107086 +0000 2024-03-23T07:55:08.820 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T07:55:08.951 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-23T07:55:08.951 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-23T07:55:08.951 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00035289 s, 1.5 MB/s 2024-03-23T07:55:08.955 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T07:55:09.009 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-23 07:54:01.245346439 +0000 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-23 07:51:25.739091528 +0000 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-23 07:51:25.739091528 +0000 2024-03-23T07:55:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-23 07:51:25.739091528 +0000 2024-03-23T07:55:09.056 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T07:55:09.185 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-23T07:55:09.185 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-23T07:55:09.185 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000325216 s, 1.6 MB/s 2024-03-23T07:55:09.189 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T07:55:09.244 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-03-23T07:55:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-23 07:54:01.246346413 +0000 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-23 07:51:26.308076674 +0000 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-23 07:51:26.308076674 +0000 2024-03-23T07:55:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-23 07:51:26.308076674 +0000 2024-03-23T07:55:09.294 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T07:55:09.376 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-23T07:55:09.376 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-23T07:55:09.376 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000304871 s, 1.7 MB/s 2024-03-23T07:55:09.380 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T07:55:09.432 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-03-23T07:55:09.481 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-23 07:54:01.246346413 +0000 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-23 07:51:26.894061377 +0000 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-23 07:51:26.894061377 +0000 2024-03-23T07:55:09.482 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-23 07:51:26.894061377 +0000 2024-03-23T07:55:09.482 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T07:55:09.568 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-23T07:55:09.568 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-23T07:55:09.568 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000347564 s, 1.5 MB/s 2024-03-23T07:55:09.574 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T07:55:09.630 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:09.630 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-23T07:55:09.630 INFO:tasks.ceph:Generating config... 2024-03-23T07:55:09.631 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-23T07:55:09.631 INFO:tasks.ceph:[client] debug client = 20 2024-03-23T07:55:09.631 INFO:tasks.ceph:[client] debug ms = 1 2024-03-23T07:55:09.632 INFO:tasks.ceph:[client] pid file = 2024-03-23T07:55:09.632 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-23T07:55:09.632 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-23T07:55:09.632 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-23T07:55:09.632 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-23T07:55:09.633 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-23T07:55:09.633 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-23T07:55:09.633 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-23T07:55:09.633 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-23T07:55:09.633 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-23T07:55:09.633 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-23T07:55:09.634 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-23T07:55:09.634 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-23T07:55:09.634 INFO:tasks.ceph:Setting up mon.a... 2024-03-23T07:55:09.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-23T07:55:09.788 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2024-03-23T07:55:09.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-23T07:55:09.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T07:55:10.002 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105'), ('mon.b', '172.21.15.132'), ('mon.c', '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]')] 2024-03-23T07:55:10.003 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-23T07:55:10.003 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:10.004 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-23T07:55:10.053 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --add b 172.21.15.132 --addv c '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid e641b836-a0e5-4e95-93aa-bb0628868335 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = pacific 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:fsid e641b836-a0e5-4e95-93aa-bb0628868335 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2024-03-23T07:55:10.203213+0000 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-03-23T07:55:10.203213+0000 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 16 (pacific) 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 1 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:1: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.b 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-23T07:55:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-23T07:55:10.205 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-23T07:55:10.252 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e641b836-a0e5-4e95-93aa-bb0628868335... 2024-03-23T07:55:10.253 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T07:55:10.257 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T07:55:10.410 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2024-03-23T07:55:10.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2024-03-23T07:55:10.412 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.105,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on skipped message = False 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = e641b836-a0e5-4e95-93aa-bb0628868335 2024-03-23T07:55:10.413 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.414 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2024-03-23T07:55:10.415 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 20 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 20 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 10 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2024-03-23T07:55:10.416 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2024-03-23T07:55:10.417 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2024-03-23T07:55:10.418 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2024-03-23T07:55:10.419 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2024-03-23T07:55:10.420 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-03-23T07:55:10.421 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-03-23T07:55:10.421 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-03-23T07:55:10.432 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-03-23T07:55:10.433 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.434 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.105,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = e641b836-a0e5-4e95-93aa-bb0628868335 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.435 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-03-23T07:55:10.437 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-23T07:55:10.438 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T07:55:10.439 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-23T07:55:10.440 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2024-03-23T07:55:10.441 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = True 2024-03-23T07:55:10.442 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-03-23T07:55:10.442 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-03-23T07:55:10.442 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-03-23T07:55:10.476 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-23T07:55:10.476 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-23T07:55:10.621 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-23T07:55:10.621 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:10.621 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-23T07:55:10.669 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:10.669 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-23T07:55:10.765 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:55:10.766 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:10.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-23T07:55:10.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T07:55:10.892 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:10.893 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:10.989 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:55:10.989 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:10.989 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-23T07:55:10.989 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T07:55:11.141 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:11.141 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:11.237 INFO:tasks.ceph:Setting up mon nodes... 2024-03-23T07:55:11.237 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-23T07:55:11.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-23T07:55:11.410 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-23T07:55:11.418 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-23T07:55:11.610 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-23T07:55:11.621 INFO:tasks.ceph:Setting up mds nodes... 2024-03-23T07:55:11.621 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-23T07:55:11.777 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-23T07:55:11.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-23T07:55:11.863 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-23T07:55:12.001 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-23T07:55:12.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-23T07:55:12.084 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-23T07:55:12.266 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-23T07:55:12.275 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-23T07:55:12.367 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-23T07:55:12.544 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-23T07:55:12.554 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-23T07:55:12.648 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-23T07:55:12.649 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T07:55:12.766 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-23T07:55:12.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-23T07:55:12.926 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-23T07:55:12.983 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-23T07:55:13.110 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-23T07:55:13.160 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-23T07:55:13.275 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-23T07:55:13.334 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-23T07:55:13.334 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-23T07:55:13.334 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-23T07:55:13.432 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:13.432 INFO:tasks.ceph:role: osd.0 2024-03-23T07:55:13.432 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:55:13.432 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-23T07:55:13.528 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:13.529 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:14.044 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-23T07:55:14.051 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-23T07:55:14.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-23T07:55:14.239 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-23T07:55:14.343 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-23T07:55:14.436 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:14.436 INFO:tasks.ceph:role: osd.1 2024-03-23T07:55:14.436 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:55:14.436 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-23T07:55:14.524 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:14.525 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:15.000 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-23T07:55:15.007 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-23T07:55:15.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-23T07:55:15.147 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-23T07:55:15.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-23T07:55:15.324 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:15.324 INFO:tasks.ceph:role: osd.2 2024-03-23T07:55:15.324 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:55:15.324 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:15.417 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:15.936 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-23T07:55:15.943 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-23T07:55:15.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-23T07:55:16.140 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-23T07:55:16.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-23T07:55:16.336 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:16.337 INFO:tasks.ceph:role: osd.3 2024-03-23T07:55:16.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:55:16.337 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-23T07:55:16.435 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:16.436 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:16.898 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-23T07:55:16.905 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-23T07:55:16.905 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-23T07:55:17.056 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-23T07:55:17.150 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:17.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:17.303+0000 7f906b3fe4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-23T07:55:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:17.303+0000 7f906b3fe4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-23T07:55:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:17.303+0000 7f906b3fe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T07:55:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:17.303+0000 7f906b3fe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T07:55:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:17.303+0000 7f906b3fe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T07:55:17.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:17.313+0000 7f906b3fe4c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-23T07:55:20.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-23T07:55:20.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:20.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:20.481+0000 7f45dac694c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-23T07:55:20.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:20.482+0000 7f45dac694c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-23T07:55:20.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:20.482+0000 7f45dac694c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T07:55:20.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:20.482+0000 7f45dac694c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T07:55:20.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:20.482+0000 7f45dac694c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T07:55:20.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:20.484+0000 7f45dac694c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-23T07:55:23.400 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-23T07:55:23.494 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:23.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:23.633+0000 7f2264aa34c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-23T07:55:23.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:23.633+0000 7f2264aa34c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-23T07:55:23.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:23.633+0000 7f2264aa34c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T07:55:23.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:23.633+0000 7f2264aa34c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T07:55:23.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:23.633+0000 7f2264aa34c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T07:55:23.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:23.635+0000 7f2264aa34c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-23T07:55:26.569 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-23T07:55:26.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:26.802+0000 7fb3daa214c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-23T07:55:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:26.803+0000 7fb3daa214c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-23T07:55:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:26.803+0000 7fb3daa214c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T07:55:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:26.803+0000 7fb3daa214c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T07:55:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:26.803+0000 7fb3daa214c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T07:55:26.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:26.805+0000 7fb3daa214c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-23T07:55:29.976 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-23T07:55:30.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-23T07:55:30.168 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:30.168 INFO:tasks.ceph:role: osd.4 2024-03-23T07:55:30.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:55:30.168 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-23T07:55:30.265 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:30.265 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:30.265 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:30.265 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:30.265 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:30.265 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:30.265 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:30.266 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:30.266 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:30.266 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:30.784 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-23T07:55:30.791 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-23T07:55:30.791 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-23T07:55:30.980 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-23T07:55:31.085 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-23T07:55:31.179 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:31.179 INFO:tasks.ceph:role: osd.5 2024-03-23T07:55:31.179 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:55:31.179 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-23T07:55:31.274 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:31.274 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:31.275 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:31.760 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-23T07:55:31.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-23T07:55:31.768 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-23T07:55:31.909 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-23T07:55:32.001 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-23T07:55:32.091 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:32.092 INFO:tasks.ceph:role: osd.6 2024-03-23T07:55:32.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:55:32.092 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:32.187 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:32.697 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-23T07:55:32.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-23T07:55:32.705 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-23T07:55:32.885 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-23T07:55:32.988 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-23T07:55:33.083 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T07:55:33.083 INFO:tasks.ceph:role: osd.7 2024-03-23T07:55:33.083 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:55:33.083 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-23T07:55:33.179 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T07:55:33.179 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T07:55:33.179 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T07:55:33.179 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-23T07:55:33.179 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T07:55:33.179 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-23T07:55:33.180 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T07:55:33.180 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-23T07:55:33.180 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T07:55:33.180 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T07:55:33.654 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-23T07:55:33.660 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-23T07:55:33.660 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-23T07:55:33.810 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-23T07:55:33.898 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:34.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:34.038+0000 7fa4a14a54c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-23T07:55:34.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:34.040+0000 7fa4a14a54c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-23T07:55:34.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:34.040+0000 7fa4a14a54c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T07:55:34.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:34.040+0000 7fa4a14a54c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T07:55:34.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:34.040+0000 7fa4a14a54c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T07:55:34.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:34.046+0000 7fa4a14a54c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-23T07:55:36.970 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-23T07:55:37.057 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:37.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:37.194+0000 7f6c152164c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-23T07:55:37.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:37.194+0000 7f6c152164c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-23T07:55:37.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:37.194+0000 7f6c152164c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T07:55:37.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:37.194+0000 7f6c152164c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T07:55:37.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:37.194+0000 7f6c152164c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T07:55:37.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:37.197+0000 7f6c152164c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-23T07:55:37.576 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:55:37.584 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:55:40.132 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-23T07:55:40.226 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:40.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:40.361+0000 7f9a860da4c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-23T07:55:40.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:40.362+0000 7f9a860da4c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-23T07:55:40.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:40.362+0000 7f9a860da4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T07:55:40.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:40.362+0000 7f9a860da4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T07:55:40.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:40.362+0000 7f9a860da4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T07:55:40.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:40.364+0000 7f9a860da4c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-23T07:55:43.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-23T07:55:43.652 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:43.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:43.783+0000 7f5dfd7f74c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-23T07:55:43.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:43.784+0000 7f5dfd7f74c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-23T07:55:43.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:43.784+0000 7f5dfd7f74c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T07:55:43.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:43.784+0000 7f5dfd7f74c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T07:55:43.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:43.784+0000 7f5dfd7f74c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T07:55:43.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:43.786+0000 7f5dfd7f74c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-23T07:55:47.215 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-23T07:55:47.308 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-23T07:55:47.308 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:47.308 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-23T07:55:47.408 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:47.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-23T07:55:47.502 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:47.502 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-23T07:55:47.597 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:47.597 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-23T07:55:47.686 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:47.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-23T07:55:47.775 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:47.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-23T07:55:47.913 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:47.913 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-23T07:55:48.009 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-23T07:55:48.109 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.110 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-23T07:55:48.201 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.202 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-23T07:55:48.297 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.297 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-23T07:55:48.389 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.390 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-23T07:55:48.483 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.483 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-23T07:55:48.566 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.566 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-23T07:55:48.706 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:48.707 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-23T07:55:48.763 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:48.763 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-23T07:55:48.814 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.814 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-23T07:55:48.868 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:48.868 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-23T07:55:48.914 INFO:tasks.ceph:Adding keys to all mons... 2024-03-23T07:55:48.915 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-23T07:55:48.922 DEBUG:teuthology.orchestra.run.smithi132:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-23T07:55:49.005 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2024-03-23T07:55:49.005 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDfiv5lHHKFGBAA1bRoBZNGoRMmDAl1UckSBw== 2024-03-23T07:55:49.005 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDfiv5lS0tvLhAA0B6QpWNT6Gv0jpwz9m7eVQ== 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDgiv5lpRAsABAAR7J8huk1V8IaPIYWVVi/Sw== 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDliv5ls98rEhAATKBcZ6oqsC7bb1yMNqg/mw== 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDoiv5l9jPMHBAA41vAhx2/0VhbdaCTrsDYKA== 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDriv5lyrDIJRAAtgl31leEL9BiyDsScyg3VA== 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDuiv5lGjHYLxAAylv6QFtPoGOi+pqqQfbErg== 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2024-03-23T07:55:49.006 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDfiv5lYK9uJBAAmspnwmPypBVVNfA6jT1s7A== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDgiv5l4RrwDxAAWuB5mTlINA3nbVSNPnEI4Q== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDgiv5lDEaKIBAAh/0WWw23V91S1a8+nGyfiA== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD2iv5lPwNfAhAAhpGoaSwdTfjRf9MMdV0h0w== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD5iv5lmeCiCxAAZSas9f18ILuB/hiBbJ3CoQ== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD8iv5lkvWfFRAAWLgyFRQb2UBjzt8SM1c4ig== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD/iv5lqtnCLhAAFQEpnFeggOL+NnTZxRsLHA== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDgiv5lYdvBLRAAvSMYjNTR3seBTqA/6zasaw== 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2024-03-23T07:55:49.007 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDgiv5llbFMNxAAYQEtkulLtGFQzIZWLRQBng== 2024-03-23T07:55:49.008 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2024-03-23T07:55:49.008 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDhiv5lmXqpBhAAFrxBNDUC6tFroTGFMPq30g== 2024-03-23T07:55:49.008 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2024-03-23T07:55:49.008 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDhiv5lLq14EBAA9u0DqAzFc2EK9fkAouya/w== 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.y] 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDfiv5lHHKFGBAA1bRoBZNGoRMmDAl1UckSBw== 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDfiv5lS0tvLhAA0B6QpWNT6Gv0jpwz9m7eVQ== 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDgiv5lpRAsABAAR7J8huk1V8IaPIYWVVi/Sw== 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.0] 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDliv5ls98rEhAATKBcZ6oqsC7bb1yMNqg/mw== 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.1] 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDoiv5l9jPMHBAA41vAhx2/0VhbdaCTrsDYKA== 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.2] 2024-03-23T07:55:49.043 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDriv5lyrDIJRAAtgl31leEL9BiyDsScyg3VA== 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.3] 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDuiv5lGjHYLxAAylv6QFtPoGOi+pqqQfbErg== 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.x] 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDfiv5lYK9uJBAAmspnwmPypBVVNfA6jT1s7A== 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDgiv5l4RrwDxAAWuB5mTlINA3nbVSNPnEI4Q== 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDgiv5lDEaKIBAAh/0WWw23V91S1a8+nGyfiA== 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.4] 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQD2iv5lPwNfAhAAhpGoaSwdTfjRf9MMdV0h0w== 2024-03-23T07:55:49.044 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.5] 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQD5iv5lmeCiCxAAZSas9f18ILuB/hiBbJ3CoQ== 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.6] 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQD8iv5lkvWfFRAAWLgyFRQb2UBjzt8SM1c4ig== 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.7] 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQD/iv5lqtnCLhAAFQEpnFeggOL+NnTZxRsLHA== 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDgiv5lYdvBLRAAvSMYjNTR3seBTqA/6zasaw== 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout:[client.2] 2024-03-23T07:55:49.045 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDgiv5llbFMNxAAYQEtkulLtGFQzIZWLRQBng== 2024-03-23T07:55:49.046 INFO:teuthology.orchestra.run.smithi132.stdout:[client.1] 2024-03-23T07:55:49.046 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDhiv5lmXqpBhAAFrxBNDUC6tFroTGFMPq30g== 2024-03-23T07:55:49.046 INFO:teuthology.orchestra.run.smithi132.stdout:[client.3] 2024-03-23T07:55:49.046 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDhiv5lLq14EBAA9u0DqAzFc2EK9fkAouya/w== 2024-03-23T07:55:49.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T07:55:49.056 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T07:55:49.196 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:49.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:49.342 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:49.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:49.492 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:49.497 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:49.633 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:49.638 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:49.770 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:49.777 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:49.918 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:49.924 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T07:55:50.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T07:55:50.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:50.208 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:50.354 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:50.361 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T07:55:50.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.510 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.654 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.661 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.805 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.945 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:50.951 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T07:55:51.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.096 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.239 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T07:55:51.694 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-23T07:55:51.694 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-23T07:55:51.779 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T07:55:51.932 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-23T07:55:52.021 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-23T07:55:52.121 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T07:55:52.273 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-23T07:55:52.358 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-23T07:55:52.448 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T07:55:52.602 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-23T07:55:52.698 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:52.705 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-23T07:55:52.759 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-23T07:55:52.759 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-23T07:55:52.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-23T07:55:52.765 INFO:tasks.ceph.mon.a:Started 2024-03-23T07:55:52.765 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-23T07:55:52.765 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-23T07:55:52.769 INFO:tasks.ceph.mon.b:Started 2024-03-23T07:55:52.770 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-23T07:55:52.770 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-23T07:55:52.774 INFO:tasks.ceph.mon.c:Started 2024-03-23T07:55:52.774 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-23T07:55:52.774 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-23T07:55:52.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-23T07:55:52.779 INFO:tasks.ceph.mgr.y:Started 2024-03-23T07:55:52.779 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-23T07:55:52.779 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-23T07:55:52.785 INFO:tasks.ceph.mgr.x:Started 2024-03-23T07:55:52.785 INFO:tasks.ceph:Setting crush tunables to default 2024-03-23T07:55:52.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2024-03-23T07:55:52.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.959+0000 7f3934a00640 1 Processor -- start 2024-03-23T07:55:52.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.959+0000 7f3934a00640 1 -- start start 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f3934a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f3934a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f3934a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300ccd90 con 0x7f393015c820 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f3934a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300cdd70 con 0x7f39300cf7c0 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f3934a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300b7fc0 con 0x7f3930160e90 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f392de00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f392f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:52.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f392f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:37288/0 (socket says 172.21.15.105:37288) 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f392f200640 1 -- 172.21.15.105:0/2965935403 learned_addr learned my addr 172.21.15.105:0/2965935403 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.960+0000 7f392de00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:37378/0 (socket says 172.21.15.105:37378) 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392f200640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 msgr2=0x7f39300d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392f200640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 msgr2=0x7f39300d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392f200640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392de00640 1 -- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 msgr2=0x7f39300cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392de00640 1 -- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 msgr2=0x7f39300cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392f200640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T07:55:52.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:52.961+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T07:55:53.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.162+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.162+0000 7f392f200640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392de00640 1 -- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 msgr2=0x7f39300cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392de00640 1 -- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 msgr2=0x7f39300cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392f200640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 msgr2=0x7f39300d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392f200640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 msgr2=0x7f39300d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392f200640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T07:55:53.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.163+0000 7f392f200640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T07:55:53.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.208+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998038182 0 0) 0x7f39300ccd90 con 0x7f393015c820 2024-03-23T07:55:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.208+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3914003520 con 0x7f393015c820 2024-03-23T07:55:53.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.214+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4138770000 0 0) 0x7f3914003520 con 0x7f393015c820 2024-03-23T07:55:53.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.214+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 msgr2=0x7f39300d0560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T07:55:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.214+0000 7f392d400640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f39300d0560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.214+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 msgr2=0x7f39300cd830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T07:55:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.214+0000 7f392d400640 1 --2- 172.21.15.105:0/2965935403 >> v2:172.21.15.132:3300/0 conn(0x7f39300cf7c0 0x7f39300cd830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.214+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300b82a0 con 0x7f393015c820 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.214+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f3920001670 con 0x7f393015c820 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 >> v1:172.21.15.132:6789/0 conn(0x7f393015c820 legacy=0x7f39300cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392d400640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f3914003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392d400640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392d400640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3914004d00 0x7f3914005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cdd70 con 0x7f3914003ff0 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300b7fc0 con 0x7f3914004d00 2024-03-23T07:55:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391400bb70 con 0x7f3930160e90 2024-03-23T07:55:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.215+0000 7f392f200640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3914004d00 0x7f3914005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.216+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.216+0000 7f392e800640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f3914003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.216+0000 7f392de00640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 msgr2=0x7f3914003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.216+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3930160e90 0x7f3914003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.216+0000 7f392de00640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3914004d00 msgr2=0x7f3914005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.216+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3914004d00 0x7f3914005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.216+0000 7f392de00640 1 -- 172.21.15.105:0/2965935403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914003520 con 0x7f3914003ff0 2024-03-23T07:55:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.218+0000 7f392de00640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39140047c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f391c00da90 tx=0x7f39140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad8bff86ce7fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.220+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c02c860 con 0x7f3914003ff0 2024-03-23T07:55:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.220+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c030070 con 0x7f3914003ff0 2024-03-23T07:55:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.220+0000 7f392d400640 1 -- 172.21.15.105:0/2965935403 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c02cc40 con 0x7f3914003ff0 2024-03-23T07:55:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.221+0000 7f3934a00640 1 -- 172.21.15.105:0/2965935403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 msgr2=0x7f39140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.221+0000 7f3934a00640 1 --2- 172.21.15.105:0/2965935403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39140047c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f391c00da90 tx=0x7f39140057f0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.221+0000 7f392f200640 1 -- 172.21.15.105:0/2965935403 reap_dead start 2024-03-23T07:55:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.221+0000 7f3934a00640 1 -- 172.21.15.105:0/2965935403 shutdown_connections 2024-03-23T07:55:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.221+0000 7f3934a00640 1 -- 172.21.15.105:0/2965935403 >> 172.21.15.105:0/2965935403 conn(0x7f393006b050 msgr2=0x7f393006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:53.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.222+0000 7f3934a00640 1 -- 172.21.15.105:0/2965935403 shutdown_connections 2024-03-23T07:55:53.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.222+0000 7f3934a00640 1 -- 172.21.15.105:0/2965935403 wait complete. 2024-03-23T07:55:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.226+0000 7f3934a00640 1 Processor -- start 2024-03-23T07:55:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.226+0000 7f3934a00640 1 -- start start 2024-03-23T07:55:53.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.227+0000 7f3934a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39301f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:53.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.227+0000 7f3934a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39300a9100 0x7f393015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f3934a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39300b3050 0x7f39300b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f3934a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393015d9a0 con 0x7f3914003ff0 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f3934a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393015dc80 con 0x7f39300a9100 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f3934a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393015df60 con 0x7f39300b3050 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f392e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39301f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f392de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39300a9100 0x7f393015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f392de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39300a9100 0x7f393015d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:37410/0 (socket says 172.21.15.105:37410) 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f392de00640 1 -- 172.21.15.105:0/3134018067 learned_addr learned my addr 172.21.15.105:0/3134018067 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:55:53.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f392f200640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39300b3050 0x7f39300b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:53.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f392de00640 1 -- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39300b3050 msgr2=0x7f39300b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.228+0000 7f392de00640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39300b3050 0x7f39300b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f392de00640 1 -- 172.21.15.105:0/3134018067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 msgr2=0x7f39301f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f392de00640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39301f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f392de00640 1 -- 172.21.15.105:0/3134018067 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393015e240 con 0x7f39300a9100 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f392e800640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39301f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f392de00640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39300a9100 0x7f393015d0a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f391c000c00 tx=0x7f391c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f3926a00640 1 -- 172.21.15.105:0/3134018067 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c041880 con 0x7f39300a9100 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f3926a00640 1 -- 172.21.15.105:0/3134018067 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c0419e0 con 0x7f39300a9100 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f3926a00640 1 -- 172.21.15.105:0/3134018067 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c041d90 con 0x7f39300a9100 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.229+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393015e520 con 0x7f39300a9100 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.230+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39300b3a10 con 0x7f39300a9100 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.230+0000 7f3926a00640 1 -- 172.21.15.105:0/3134018067 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f391c03f740 con 0x7f39300a9100 2024-03-23T07:55:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.230+0000 7f3926a00640 1 -- 172.21.15.105:0/3134018067 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f391c04a6d0 con 0x7f39300a9100 2024-03-23T07:55:53.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.231+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3930159400 con 0x7f39300a9100 2024-03-23T07:55:53.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.235+0000 7f3926a00640 1 -- 172.21.15.105:0/3134018067 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f391c045020 con 0x7f39300a9100 2024-03-23T07:55:53.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.308+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f393015ddc0 con 0x7f39300a9100 2024-03-23T07:55:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.316+0000 7f3926a00640 1 -- 172.21.15.105:0/3134018067 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f393015ddc0 con 0x7f39300a9100 2024-03-23T07:55:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39300a9100 msgr2=0x7f393015d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39300a9100 0x7f393015d0a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f391c000c00 tx=0x7f391c0046c0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 shutdown_connections 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39300b3050 0x7f39300b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39300a9100 0x7f393015d0a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 --2- 172.21.15.105:0/3134018067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3914003ff0 0x7f39301f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 >> 172.21.15.105:0/3134018067 conn(0x7f393006b050 msgr2=0x7f393006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 shutdown_connections 2024-03-23T07:55:53.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:53.317+0000 7f3934a00640 1 -- 172.21.15.105:0/3134018067 wait complete. 2024-03-23T07:55:53.335 INFO:tasks.ceph:check_enable_crimson: False 2024-03-23T07:55:53.335 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-23T07:55:53.336 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:53.336 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-23T07:55:53.416 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:53.416 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-23T07:55:53.543 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:53.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-23T07:55:53.667 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:55:53.667 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-23T07:55:53.778 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:53.778+0000 7fb386753040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T07:55:53.800 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:53.800 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-23T07:55:53.809 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:53.810+0000 7f181042e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T07:55:53.843 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:53.842+0000 7fb386753040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T07:55:53.877 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:53.877+0000 7f181042e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T07:55:53.893 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:53.893 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-23T07:55:54.022 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:54.022 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-23T07:55:54.151 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:55:54.152 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-23T07:55:54.278 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 1e926019-a0a8-4e10-b0c3-0fb0c20a45d5 0 2024-03-23T07:55:54.437 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.436+0000 7fb386753040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T07:55:54.498 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.498+0000 7fb386753040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T07:55:54.504 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:54.504+0000 7f181042e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T07:55:54.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.513+0000 7fb899000640 1 Processor -- start 2024-03-23T07:55:54.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.513+0000 7fb899000640 1 -- start start 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.514+0000 7fb899000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8940cc070 0x7fb894069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.514+0000 7fb899000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89415e540 con 0x7fb89415dd40 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb899000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb894072330 con 0x7fb8940cc4e0 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb899000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8940af0e0 con 0x7fb8940cc070 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb892800640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb8940cc4e0 0x7fb89415e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38582/0 (socket says 172.21.15.132:38582) 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb892800640 1 -- 172.21.15.132:0/4020817193 learned_addr learned my addr 172.21.15.132:0/4020817193 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb891400640 1 --2- 172.21.15.132:0/4020817193 >> v2:172.21.15.105:3300/0 conn(0x7fb8940cc070 0x7fb894069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285430113 0 0) 0x7fb894072330 con 0x7fb8940cc4e0 2024-03-23T07:55:54.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb878003520 con 0x7fb8940cc4e0 2024-03-23T07:55:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2423454942 0 0) 0x7fb878003520 con 0x7fb8940cc4e0 2024-03-23T07:55:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 >> v2:172.21.15.105:3300/0 conn(0x7fb8940cc070 msgr2=0x7fb894069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.515+0000 7fb890a00640 1 --2- 172.21.15.132:0/4020817193 >> v2:172.21.15.105:3300/0 conn(0x7fb8940cc070 0x7fb894069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 >> v1:172.21.15.105:6789/0 conn(0x7fb89415dd40 legacy=0x7fb894071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940cea20 con 0x7fb8940cc4e0 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fb88c0019f0 con 0x7fb8940cc4e0 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 >> v1:172.21.15.132:6789/0 conn(0x7fb8940cc4e0 legacy=0x7fb89415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb878003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8780047c0 0x7fb878004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb878005180 0x7fb87800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940af0e0 con 0x7fb878005180 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87800c020 con 0x7fb878003970 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87800c1e0 con 0x7fb8780047c0 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb891e00640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb878003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb891400640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8780047c0 0x7fb878004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb892800640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb878005180 0x7fb87800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb891e00640 1 -- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8780047c0 msgr2=0x7fb878004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb891e00640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8780047c0 0x7fb878004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb891e00640 1 -- 172.21.15.132:0/4020817193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb878005180 msgr2=0x7fb87800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:54.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb891e00640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb878005180 0x7fb87800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb891e00640 1 -- 172.21.15.132:0/4020817193 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878003520 con 0x7fb878003970 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb892800640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb878005180 0x7fb87800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.516+0000 7fb892800640 1 -- 172.21.15.132:0/4020817193 reap_dead start 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.517+0000 7fb891e00640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb878003d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb884001750 tx=0x7fb8780057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d8c4a2874a64bf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840052d0 con 0x7fb878003970 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88402e050 con 0x7fb878003970 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb890a00640 1 -- 172.21.15.132:0/4020817193 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884003830 con 0x7fb878003970 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb899000640 1 -- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 msgr2=0x7fb878003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb899000640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb878003d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb884001750 tx=0x7fb8780057f0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb899000640 1 -- 172.21.15.132:0/4020817193 shutdown_connections 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb899000640 1 --2- 172.21.15.132:0/4020817193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb878003d90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb899000640 1 -- 172.21.15.132:0/4020817193 >> 172.21.15.132:0/4020817193 conn(0x7fb89406d440 msgr2=0x7fb89406d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:54.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb899000640 1 -- 172.21.15.132:0/4020817193 shutdown_connections 2024-03-23T07:55:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.518+0000 7fb899000640 1 -- 172.21.15.132:0/4020817193 wait complete. 2024-03-23T07:55:54.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.520+0000 7fb899000640 1 Processor -- start 2024-03-23T07:55:54.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.520+0000 7fb899000640 1 -- start start 2024-03-23T07:55:54.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb899000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb894154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb899000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8940cb440 0x7fb8940ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb899000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb894157020 0x7fb8940ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb899000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb894158190 con 0x7fb894157020 2024-03-23T07:55:54.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb899000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940abc20 con 0x7fb878003970 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb899000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb894158430 con 0x7fb8940cb440 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb891e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb894154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb891400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8940cb440 0x7fb8940ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb891e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb894154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45586/0 (socket says 172.21.15.132:45586) 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.521+0000 7fb891e00640 1 -- 172.21.15.132:0/676149937 learned_addr learned my addr 172.21.15.132:0/676149937 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb892800640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb894157020 0x7fb8940ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb891e00640 1 -- 172.21.15.132:0/676149937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8940cb440 msgr2=0x7fb8940ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb891e00640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8940cb440 0x7fb8940ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb891e00640 1 -- 172.21.15.132:0/676149937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb894157020 msgr2=0x7fb8940ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb891e00640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb894157020 0x7fb8940ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb891e00640 1 -- 172.21.15.132:0/676149937 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb894158710 con 0x7fb878003970 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb892800640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb894157020 0x7fb8940ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb891e00640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb894154110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb88402bc00 tx=0x7fb884001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb88a000640 1 -- 172.21.15.132:0/676149937 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840076c0 con 0x7fb878003970 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb88a000640 1 -- 172.21.15.132:0/676149937 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884007a30 con 0x7fb878003970 2024-03-23T07:55:54.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8940abf00 con 0x7fb878003970 2024-03-23T07:55:54.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.522+0000 7fb88a000640 1 -- 172.21.15.132:0/676149937 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884006c20 con 0x7fb878003970 2024-03-23T07:55:54.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.523+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8940ab8b0 con 0x7fb878003970 2024-03-23T07:55:54.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.523+0000 7fb88a000640 1 -- 172.21.15.132:0/676149937 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb8840038f0 con 0x7fb878003970 2024-03-23T07:55:54.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.523+0000 7fb88a000640 1 -- 172.21.15.132:0/676149937 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb88403d7e0 con 0x7fb878003970 2024-03-23T07:55:54.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.524+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb894066bb0 con 0x7fb878003970 2024-03-23T07:55:54.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.526+0000 7fb88a000640 1 -- 172.21.15.132:0/676149937 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fb884041020 con 0x7fb878003970 2024-03-23T07:55:54.550 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.549+0000 7fb386753040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T07:55:54.569 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:54.569+0000 7f181042e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T07:55:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.604+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1e926019-a0a8-4e10-b0c3-0fb0c20a45d5", "id": 0} v 0) v1 -- 0x7fb894071520 con 0x7fb878003970 2024-03-23T07:55:54.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.611+0000 7fb88a000640 1 -- 172.21.15.132:0/676149937 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1e926019-a0a8-4e10-b0c3-0fb0c20a45d5", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb884001370 con 0x7fb878003970 2024-03-23T07:55:54.611 INFO:teuthology.orchestra.run.smithi132.stdout:0 2024-03-23T07:55:54.611 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.611+0000 7fb386753040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T07:55:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 msgr2=0x7fb894154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb894154110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb88402bc00 tx=0x7fb884001120 comp rx=0 tx=0).stop 2024-03-23T07:55:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 shutdown_connections 2024-03-23T07:55:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb894157020 0x7fb8940ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8940cb440 0x7fb8940ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 --2- 172.21.15.132:0/676149937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb878003970 0x7fb894154110 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 >> 172.21.15.132:0/676149937 conn(0x7fb89406d440 msgr2=0x7fb8941563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.612+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 shutdown_connections 2024-03-23T07:55:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.613+0000 7fb899000640 1 -- 172.21.15.132:0/676149937 wait complete. 2024-03-23T07:55:54.623 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:54.623+0000 7f181042e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T07:55:54.627 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 7f82c10f-8491-4200-bf31-48a058dacfa2 1 2024-03-23T07:55:54.660 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.659+0000 7fb386753040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T07:55:54.690 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:54.690+0000 7f181042e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T07:55:54.742 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:54.743+0000 7f181042e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T07:55:54.787 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.786+0000 7fb386753040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T07:55:54.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.811+0000 7f01b2e00640 1 Processor -- start 2024-03-23T07:55:54.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.811+0000 7f01b2e00640 1 -- start start 2024-03-23T07:55:54.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.812+0000 7f01b2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac15cd10 0x7f01ac0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.812+0000 7f01b2e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f01ac0b47d0 0x7f01ac0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01b2e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac1617d0 con 0x7f01ac06b030 2024-03-23T07:55:54.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01b2e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac0b5150 con 0x7f01ac0b47d0 2024-03-23T07:55:54.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01b2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac160c50 con 0x7f01ac15cd10 2024-03-23T07:55:54.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac15cd10 0x7f01ac0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01ab400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f01ac0b47d0 0x7f01ac0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac15cd10 0x7f01ac0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:36210/0 (socket says 172.21.15.132:36210) 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01ab400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f01ac0b47d0 0x7f01ac0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45590/0 (socket says 172.21.15.132:45590) 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01abe00640 1 -- 172.21.15.132:0/2825591936 learned_addr learned my addr 172.21.15.132:0/2825591936 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01ab400640 1 -- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac15cd10 msgr2=0x7f01ac0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01ab400640 1 --2- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac15cd10 0x7f01ac0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01ab400640 1 -- 172.21.15.132:0/2825591936 >> v1:172.21.15.105:6789/0 conn(0x7f01ac06b030 legacy=0x7f01ac06b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01ab400640 1 -- 172.21.15.132:0/2825591936 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac160f30 con 0x7f01ac0b47d0 2024-03-23T07:55:54.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.813+0000 7f01abe00640 1 --2- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac15cd10 0x7f01ac0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:54.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.814+0000 7f01ab400640 1 --2- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b4c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01980097a0 tx=0x7f019802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe51a03f489f386 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:54.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01aaa00640 1 -- 172.21.15.132:0/2825591936 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198031780 con 0x7f01ac0b47d0 2024-03-23T07:55:54.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01aaa00640 1 -- 172.21.15.132:0/2825591936 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01980318e0 con 0x7f01ac0b47d0 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01aaa00640 1 -- 172.21.15.132:0/2825591936 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198031c70 con 0x7f01ac0b47d0 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 -- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 msgr2=0x7f01ac0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 --2- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b4c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01980097a0 tx=0x7f019802b820 comp rx=0 tx=0).stop 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 -- 172.21.15.132:0/2825591936 shutdown_connections 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 --2- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b4c10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 --2- 172.21.15.132:0/2825591936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac15cd10 0x7f01ac0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 -- 172.21.15.132:0/2825591936 >> 172.21.15.132:0/2825591936 conn(0x7f01ac0cf830 msgr2=0x7f01ac0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 -- 172.21.15.132:0/2825591936 shutdown_connections 2024-03-23T07:55:54.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.815+0000 7f01b2e00640 1 -- 172.21.15.132:0/2825591936 wait complete. 2024-03-23T07:55:54.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.817+0000 7f01b2e00640 1 Processor -- start 2024-03-23T07:55:54.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.817+0000 7f01b2e00640 1 -- start start 2024-03-23T07:55:54.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01b2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac06b030 0x7f01ac15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01b2e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01b2e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac15cd10 0x7f01ac156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01b2e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0c6300 con 0x7f01ac15cd10 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01b2e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0c65e0 con 0x7f01ac0b47d0 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01b2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0c68c0 con 0x7f01ac06b030 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01ab400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac06b030 0x7f01ac15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01ab400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45604/0 (socket says 172.21.15.132:45604) 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.818+0000 7f01ab400640 1 -- 172.21.15.132:0/244335562 learned_addr learned my addr 172.21.15.132:0/244335562 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01b0e00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac15cd10 0x7f01ac156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01abe00640 1 -- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 msgr2=0x7f01ac0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01abe00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01abe00640 1 -- 172.21.15.132:0/244335562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac15cd10 msgr2=0x7f01ac156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01abe00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac15cd10 0x7f01ac156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01abe00640 1 -- 172.21.15.132:0/244335562 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0094d0 con 0x7f01ac06b030 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01ab400640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01b0e00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac15cd10 0x7f01ac156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01abe00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac06b030 0x7f01ac15d4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019c00e900 tx=0x7f019c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01a3e00640 1 -- 172.21.15.132:0/244335562 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c00ca00 con 0x7f01ac06b030 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01a3e00640 1 -- 172.21.15.132:0/244335562 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c00cb60 con 0x7f01ac06b030 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01a3e00640 1 -- 172.21.15.132:0/244335562 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c004030 con 0x7f01ac06b030 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0198009400 con 0x7f01ac06b030 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.819+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac0c6e80 con 0x7f01ac06b030 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.820+0000 7f01a3e00640 1 -- 172.21.15.132:0/244335562 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f019c005990 con 0x7f01ac06b030 2024-03-23T07:55:54.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.820+0000 7f01a3e00640 1 -- 172.21.15.132:0/244335562 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f019c003df0 con 0x7f01ac06b030 2024-03-23T07:55:54.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.821+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac0ce180 con 0x7f01ac06b030 2024-03-23T07:55:54.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.823+0000 7f01a3e00640 1 -- 172.21.15.132:0/244335562 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f019c020060 con 0x7f01ac06b030 2024-03-23T07:55:54.867 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.866+0000 7fb386753040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T07:55:54.879 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:54.879+0000 7f181042e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T07:55:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.902+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7f82c10f-8491-4200-bf31-48a058dacfa2", "id": 1} v 0) v1 -- 0x7f01ac0cef60 con 0x7f01ac06b030 2024-03-23T07:55:54.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.909+0000 7f01a3e00640 1 -- 172.21.15.132:0/244335562 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7f82c10f-8491-4200-bf31-48a058dacfa2", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f019c005e20 con 0x7f01ac06b030 2024-03-23T07:55:54.910 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-03-23T07:55:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac06b030 msgr2=0x7f01ac15d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac06b030 0x7f01ac15d4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019c00e900 tx=0x7f019c00edc0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 shutdown_connections 2024-03-23T07:55:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac15cd10 0x7f01ac156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01ac0b47d0 0x7f01ac0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 --2- 172.21.15.132:0/244335562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01ac06b030 0x7f01ac15d4f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 >> 172.21.15.132:0/244335562 conn(0x7f01ac0cf830 msgr2=0x7f01ac0cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 shutdown_connections 2024-03-23T07:55:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:54.911+0000 7f01b2e00640 1 -- 172.21.15.132:0/244335562 wait complete. 2024-03-23T07:55:54.920 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.919+0000 7fb386753040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T07:55:54.928 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new c61d9003-c7ec-476e-9bc9-0c1fa7939fda 2 2024-03-23T07:55:54.965 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:54.965+0000 7f181042e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T07:55:54.979 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:54.978+0000 7fb386753040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T07:55:55.022 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:55.022+0000 7f181042e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T07:55:55.033 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:55.032+0000 7fb386753040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T07:55:55.085 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:55.085+0000 7f181042e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T07:55:55.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.103+0000 7fe0b5600640 1 Processor -- start 2024-03-23T07:55:55.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.103+0000 7fe0b5600640 1 -- start start 2024-03-23T07:55:55.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.104+0000 7fe0b5600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe0b00d0d40 0x7fe0b00b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.104+0000 7fe0b5600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00d20c0 con 0x7fe0b00b84b0 2024-03-23T07:55:55.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.104+0000 7fe0b5600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00b8940 con 0x7fe0b00cd740 2024-03-23T07:55:55.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.104+0000 7fe0b5600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b00d14b0 con 0x7fe0b00d0d40 2024-03-23T07:55:55.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.104+0000 7fe0aea00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe0b00cd740 0x7fe0b00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38590/0 (socket says 172.21.15.132:38590) 2024-03-23T07:55:55.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.104+0000 7fe0aea00640 1 -- 172.21.15.132:0/2798547598 learned_addr learned my addr 172.21.15.132:0/2798547598 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-23T07:55:55.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.105+0000 7fe0af400640 1 --2- 172.21.15.132:0/2798547598 >> v2:172.21.15.132:3300/0 conn(0x7fe0b00d0d40 0x7fe0b00b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.105+0000 7fe0ae000640 1 -- 172.21.15.132:0/2798547598 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977643779 0 0) 0x7fe0b00b8940 con 0x7fe0b00cd740 2024-03-23T07:55:55.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.105+0000 7fe0ae000640 1 -- 172.21.15.132:0/2798547598 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe094003520 con 0x7fe0b00cd740 2024-03-23T07:55:55.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.105+0000 7fe0af400640 1 -- 172.21.15.132:0/2798547598 >> v1:172.21.15.132:6789/0 conn(0x7fe0b00cd740 legacy=0x7fe0b00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.105+0000 7fe0af400640 1 -- 172.21.15.132:0/2798547598 >> v1:172.21.15.105:6789/0 conn(0x7fe0b00b84b0 legacy=0x7fe0b00b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.105+0000 7fe0af400640 1 -- 172.21.15.132:0/2798547598 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00d1790 con 0x7fe0b00d0d40 2024-03-23T07:55:55.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.106+0000 7fe0af400640 1 --2- 172.21.15.132:0/2798547598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00d0d40 0x7fe0b00b61b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009d90 tx=0x7fe0a802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41077c0d20a5cc0f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:55.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.106+0000 7fe0ae000640 1 -- 172.21.15.132:0/2798547598 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a802cd00 con 0x7fe0b00d0d40 2024-03-23T07:55:55.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.106+0000 7fe0ae000640 1 -- 172.21.15.132:0/2798547598 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a802ce60 con 0x7fe0b00d0d40 2024-03-23T07:55:55.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.106+0000 7fe0ae000640 1 -- 172.21.15.132:0/2798547598 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8034660 con 0x7fe0b00d0d40 2024-03-23T07:55:55.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.107+0000 7fe0b5600640 1 -- 172.21.15.132:0/2798547598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00d0d40 msgr2=0x7fe0b00b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.107+0000 7fe0b5600640 1 --2- 172.21.15.132:0/2798547598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00d0d40 0x7fe0b00b61b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009d90 tx=0x7fe0a802bf50 comp rx=0 tx=0).stop 2024-03-23T07:55:55.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.107+0000 7fe0b5600640 1 -- 172.21.15.132:0/2798547598 shutdown_connections 2024-03-23T07:55:55.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.107+0000 7fe0b5600640 1 --2- 172.21.15.132:0/2798547598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00d0d40 0x7fe0b00b61b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.107+0000 7fe0b5600640 1 -- 172.21.15.132:0/2798547598 >> 172.21.15.132:0/2798547598 conn(0x7fe0b00cf5d0 msgr2=0x7fe0b00cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:55.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.107+0000 7fe0b5600640 1 -- 172.21.15.132:0/2798547598 shutdown_connections 2024-03-23T07:55:55.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.107+0000 7fe0b5600640 1 -- 172.21.15.132:0/2798547598 wait complete. 2024-03-23T07:55:55.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.109+0000 7fe0b5600640 1 Processor -- start 2024-03-23T07:55:55.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.109+0000 7fe0b5600640 1 -- start start 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0b5600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00b84b0 0x7fe0b00aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0b5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0b00cd740 0x7fe0b00acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0b5600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0b00d0d40 0x7fe0b00ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0b5600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00aa710 con 0x7fe0b00d0d40 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0b5600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b014e880 con 0x7fe0b00b84b0 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0b5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b014eb60 con 0x7fe0b00cd740 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0a7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0b00cd740 0x7fe0b00acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0aea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00b84b0 0x7fe0b00aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0a7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0b00cd740 0x7fe0b00acbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:36232/0 (socket says 172.21.15.132:36232) 2024-03-23T07:55:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0a7e00640 1 -- 172.21.15.132:0/377289406 learned_addr learned my addr 172.21.15.132:0/377289406 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0af400640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0b00d0d40 0x7fe0b00ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0a7e00640 1 -- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00b84b0 msgr2=0x7fe0b00aa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0a7e00640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00b84b0 0x7fe0b00aa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.110+0000 7fe0a7e00640 1 -- 172.21.15.132:0/377289406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0b00d0d40 msgr2=0x7fe0b00ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a7e00640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0b00d0d40 0x7fe0b00ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a7e00640 1 -- 172.21.15.132:0/377289406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a8009a70 con 0x7fe0b00cd740 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0aea00640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00b84b0 0x7fe0b00aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a7e00640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0b00cd740 0x7fe0b00acbb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c450 tx=0x7fe09800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a6a00640 1 -- 172.21.15.132:0/377289406 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980052d0 con 0x7fe0b00cd740 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a6a00640 1 -- 172.21.15.132:0/377289406 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe098005430 con 0x7fe0b00cd740 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a6a00640 1 -- 172.21.15.132:0/377289406 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098012870 con 0x7fe0b00cd740 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b014ee40 con 0x7fe0b00cd740 2024-03-23T07:55:55.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b014f400 con 0x7fe0b00cd740 2024-03-23T07:55:55.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a6a00640 1 -- 172.21.15.132:0/377289406 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe098012cd0 con 0x7fe0b00cd740 2024-03-23T07:55:55.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.111+0000 7fe0a6a00640 1 -- 172.21.15.132:0/377289406 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fe098003a10 con 0x7fe0b00cd740 2024-03-23T07:55:55.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.113+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b00cacf0 con 0x7fe0b00cd740 2024-03-23T07:55:55.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.115+0000 7fe0a6a00640 1 -- 172.21.15.132:0/377289406 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fe0b00cacf0 con 0x7fe0b00cd740 2024-03-23T07:55:55.144 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:55.144+0000 7f181042e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T07:55:55.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.192+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c61d9003-c7ec-476e-9bc9-0c1fa7939fda", "id": 2} v 0) v1 -- 0x7fe0b0073390 con 0x7fe0b00cd740 2024-03-23T07:55:55.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.202+0000 7fe0a6a00640 1 -- 172.21.15.132:0/377289406 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c61d9003-c7ec-476e-9bc9-0c1fa7939fda", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0980035f0 con 0x7fe0b00cd740 2024-03-23T07:55:55.202 INFO:teuthology.orchestra.run.smithi132.stdout:2 2024-03-23T07:55:55.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0b00cd740 msgr2=0x7fe0b00acbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0b00cd740 0x7fe0b00acbb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c450 tx=0x7fe09800c910 comp rx=0 tx=0).stop 2024-03-23T07:55:55.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 shutdown_connections 2024-03-23T07:55:55.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0b00d0d40 0x7fe0b00ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0b00cd740 0x7fe0b00acbb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 --2- 172.21.15.132:0/377289406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0b00b84b0 0x7fe0b00aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 >> 172.21.15.132:0/377289406 conn(0x7fe0b00cf5d0 msgr2=0x7fe0b00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:55.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 shutdown_connections 2024-03-23T07:55:55.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.204+0000 7fe0b5600640 1 -- 172.21.15.132:0/377289406 wait complete. 2024-03-23T07:55:55.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new f3b45192-5bab-4797-9563-205de52e2cda 3 2024-03-23T07:55:55.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.455+0000 7f6156800640 1 Processor -- start 2024-03-23T07:55:55.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.455+0000 7f6156800640 1 -- start start 2024-03-23T07:55:55.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6156800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61500d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6156800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61500b62a0 0x7f61500cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6156800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6150160ae0 con 0x7f61500d1be0 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6156800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61500d0c00 con 0x7f61500b62a0 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6156800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61500cdde0 con 0x7f615015cc30 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6154a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f61500d1be0 0x7f61500b5b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38598/0 (socket says 172.21.15.132:38598) 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6155400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61500d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.457+0000 7f6154a00640 1 -- 172.21.15.132:0/1011169424 learned_addr learned my addr 172.21.15.132:0/1011169424 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.458+0000 7f6155e00640 1 --2- 172.21.15.132:0/1011169424 >> v2:172.21.15.105:3300/0 conn(0x7f61500b62a0 0x7f61500cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.458+0000 7f614fe00640 1 -- 172.21.15.132:0/1011169424 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213433115 0 0) 0x7f6150160ae0 con 0x7f61500d1be0 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.458+0000 7f6155400640 1 -- 172.21.15.132:0/1011169424 >> v2:172.21.15.105:3300/0 conn(0x7f61500b62a0 msgr2=0x7f61500cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:55.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.458+0000 7f6155400640 1 --2- 172.21.15.132:0/1011169424 >> v2:172.21.15.105:3300/0 conn(0x7f61500b62a0 0x7f61500cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.458+0000 7f6155400640 1 -- 172.21.15.132:0/1011169424 >> v1:172.21.15.132:6789/0 conn(0x7f61500d1be0 legacy=0x7f61500b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.458+0000 7f6155400640 1 -- 172.21.15.132:0/1011169424 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61500d0ea0 con 0x7f615015cc30 2024-03-23T07:55:55.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.459+0000 7f6155400640 1 --2- 172.21.15.132:0/1011169424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61500d16a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61440097a0 tx=0x7f614402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cbf62a43eea4c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:55.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.460+0000 7f614fe00640 1 -- 172.21.15.132:0/1011169424 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144031830 con 0x7f615015cc30 2024-03-23T07:55:55.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.460+0000 7f614fe00640 1 -- 172.21.15.132:0/1011169424 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6144031990 con 0x7f615015cc30 2024-03-23T07:55:55.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.460+0000 7f614fe00640 1 -- 172.21.15.132:0/1011169424 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144031d20 con 0x7f615015cc30 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.460+0000 7f6156800640 1 -- 172.21.15.132:0/1011169424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 msgr2=0x7f61500d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.460+0000 7f6156800640 1 --2- 172.21.15.132:0/1011169424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61500d16a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61440097a0 tx=0x7f614402b8d0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.461+0000 7f6156800640 1 -- 172.21.15.132:0/1011169424 shutdown_connections 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.461+0000 7f6156800640 1 --2- 172.21.15.132:0/1011169424 >> v2:172.21.15.105:3300/0 conn(0x7f61500b62a0 0x7f61500cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.461+0000 7f6156800640 1 --2- 172.21.15.132:0/1011169424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61500d16a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.461+0000 7f6156800640 1 -- 172.21.15.132:0/1011169424 >> 172.21.15.132:0/1011169424 conn(0x7f615006b100 msgr2=0x7f615006b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.461+0000 7f6156800640 1 -- 172.21.15.132:0/1011169424 shutdown_connections 2024-03-23T07:55:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.461+0000 7f6156800640 1 -- 172.21.15.132:0/1011169424 wait complete. 2024-03-23T07:55:55.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.463+0000 7f6156800640 1 Processor -- start 2024-03-23T07:55:55.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.463+0000 7f6156800640 1 -- start start 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.464+0000 7f6156800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61500b62a0 0x7f615015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.464+0000 7f6156800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61500d1be0 0x7f6150156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.464+0000 7f6156800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61501f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.464+0000 7f6156800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61501554f0 con 0x7f61500b62a0 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.464+0000 7f6156800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6150157500 con 0x7f61500d1be0 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.464+0000 7f6156800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61501f5d30 con 0x7f615015cc30 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61500d1be0 0x7f6150156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6155e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61501f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61500d1be0 0x7f6150156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45626/0 (socket says 172.21.15.132:45626) 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 -- 172.21.15.132:0/1416653835 learned_addr learned my addr 172.21.15.132:0/1416653835 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6155400640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61500b62a0 0x7f615015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 -- 172.21.15.132:0/1416653835 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 msgr2=0x7f61501f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61501f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 -- 172.21.15.132:0/1416653835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61500b62a0 msgr2=0x7f615015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61500b62a0 0x7f615015fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.465+0000 7f6154a00640 1 -- 172.21.15.132:0/1416653835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144009400 con 0x7f61500d1be0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f6154a00640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61500d1be0 0x7f6150156fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f613c00c4d0 tx=0x7f613c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f614d600640 1 -- 172.21.15.132:0/1416653835 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c0074e0 con 0x7f61500d1be0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f614d600640 1 -- 172.21.15.132:0/1416653835 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c007660 con 0x7f61500d1be0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61501f6000 con 0x7f61500d1be0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f614d600640 1 -- 172.21.15.132:0/1416653835 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c004a20 con 0x7f61500d1be0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61501f65c0 con 0x7f61500d1be0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f614d600640 1 -- 172.21.15.132:0/1416653835 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f613c01c3b0 con 0x7f61500d1be0 2024-03-23T07:55:55.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.466+0000 7f614d600640 1 -- 172.21.15.132:0/1416653835 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f613c01caf0 con 0x7f61500d1be0 2024-03-23T07:55:55.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.467+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615004e030 con 0x7f61500d1be0 2024-03-23T07:55:55.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.470+0000 7f614d600640 1 -- 172.21.15.132:0/1416653835 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f613c018070 con 0x7f61500d1be0 2024-03-23T07:55:55.476 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:55.475+0000 7fb386753040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T07:55:55.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.547+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f3b45192-5bab-4797-9563-205de52e2cda", "id": 3} v 0) v1 -- 0x7f61500d0710 con 0x7f61500d1be0 2024-03-23T07:55:55.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.558+0000 7f614d600640 1 -- 172.21.15.132:0/1416653835 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f3b45192-5bab-4797-9563-205de52e2cda", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f613c01cdc0 con 0x7f61500d1be0 2024-03-23T07:55:55.558 INFO:teuthology.orchestra.run.smithi132.stdout:3 2024-03-23T07:55:55.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.559+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61500d1be0 msgr2=0x7f6150156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.559+0000 7f6156800640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61500d1be0 0x7f6150156fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f613c00c4d0 tx=0x7f613c00c990 comp rx=0 tx=0).stop 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.560+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 shutdown_connections 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.560+0000 7f6156800640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615015cc30 0x7f61501f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.560+0000 7f6156800640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61500d1be0 0x7f6150156fc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.560+0000 7f6156800640 1 --2- 172.21.15.132:0/1416653835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61500b62a0 0x7f615015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.560+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 >> 172.21.15.132:0/1416653835 conn(0x7f615006b100 msgr2=0x7f61500c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.560+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 shutdown_connections 2024-03-23T07:55:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.560+0000 7f6156800640 1 -- 172.21.15.132:0/1416653835 wait complete. 2024-03-23T07:55:55.575 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new d1bd131a-59ed-4ac8-b3ff-cd7074eb8387 4 2024-03-23T07:55:55.627 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:55.627+0000 7f181042e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T07:55:55.633 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:55.632+0000 7fb386753040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T07:55:55.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.759+0000 7f9712e00640 1 Processor -- start 2024-03-23T07:55:55.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.759+0000 7f9712e00640 1 -- start start 2024-03-23T07:55:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9712e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f970c06f0f0 0x7f970c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9712e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c15d480 0x7f970c166530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9712e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970c0cebc0 con 0x7f970c15cb60 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9712e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970c0b22b0 con 0x7f970c06f0f0 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9712e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0ce340 con 0x7f970c15d480 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9703e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f970c06f0f0 0x7f970c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9710e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c15d480 0x7f970c166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9703e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f970c06f0f0 0x7f970c15cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45642/0 (socket says 172.21.15.132:45642) 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9703e00640 1 -- 172.21.15.132:0/1667809349 learned_addr learned my addr 172.21.15.132:0/1667809349 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9710e00640 1 -- 172.21.15.132:0/1667809349 >> v2:172.21.15.132:3300/0 conn(0x7f970c06f0f0 msgr2=0x7f970c15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9710e00640 1 --2- 172.21.15.132:0/1667809349 >> v2:172.21.15.132:3300/0 conn(0x7f970c06f0f0 0x7f970c15cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9710e00640 1 -- 172.21.15.132:0/1667809349 >> v1:172.21.15.105:6789/0 conn(0x7f970c15cb60 legacy=0x7f970c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9710e00640 1 -- 172.21.15.132:0/1667809349 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c15d8b0 con 0x7f970c15d480 2024-03-23T07:55:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.761+0000 7f9703e00640 1 --2- 172.21.15.132:0/1667809349 >> v2:172.21.15.132:3300/0 conn(0x7f970c06f0f0 0x7f970c15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.762+0000 7f9710e00640 1 --2- 172.21.15.132:0/1667809349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c15d480 0x7f970c166530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9704009d90 tx=0x7f970402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c1d9e3d28c230b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f970b400640 1 -- 172.21.15.132:0/1667809349 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970402cbd0 con 0x7f970c15d480 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f970b400640 1 -- 172.21.15.132:0/1667809349 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970402cd30 con 0x7f970c15d480 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f970b400640 1 -- 172.21.15.132:0/1667809349 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970402cea0 con 0x7f970c15d480 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 -- 172.21.15.132:0/1667809349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c15d480 msgr2=0x7f970c166530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 --2- 172.21.15.132:0/1667809349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c15d480 0x7f970c166530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9704009d90 tx=0x7f970402be20 comp rx=0 tx=0).stop 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 -- 172.21.15.132:0/1667809349 shutdown_connections 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 --2- 172.21.15.132:0/1667809349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c15d480 0x7f970c166530 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 --2- 172.21.15.132:0/1667809349 >> v2:172.21.15.132:3300/0 conn(0x7f970c06f0f0 0x7f970c15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 -- 172.21.15.132:0/1667809349 >> 172.21.15.132:0/1667809349 conn(0x7f970c0ced50 msgr2=0x7f970c0cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:55.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 -- 172.21.15.132:0/1667809349 shutdown_connections 2024-03-23T07:55:55.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.763+0000 7f9712e00640 1 -- 172.21.15.132:0/1667809349 wait complete. 2024-03-23T07:55:55.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.765+0000 7f9712e00640 1 Processor -- start 2024-03-23T07:55:55.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.765+0000 7f9712e00640 1 -- start start 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9712e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c06f0f0 0x7f970c0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9712e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f970c15cb60 0x7f970c0ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9712e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f970c15d480 0x7f970c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9712e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0c4ea0 con 0x7f970c15cb60 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9712e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0c5180 con 0x7f970c15d480 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9712e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0c5460 con 0x7f970c06f0f0 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9710e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f970c15d480 0x7f970c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f970be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c06f0f0 0x7f970c0c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9710e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f970c15d480 0x7f970c14d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45652/0 (socket says 172.21.15.132:45652) 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.766+0000 7f9710e00640 1 -- 172.21.15.132:0/4003234137 learned_addr learned my addr 172.21.15.132:0/4003234137 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9703e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f970c15cb60 0x7f970c0ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:55.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9710e00640 1 -- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c06f0f0 msgr2=0x7f970c0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9710e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c06f0f0 0x7f970c0c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9710e00640 1 -- 172.21.15.132:0/4003234137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f970c15cb60 msgr2=0x7f970c0ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9710e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f970c15cb60 0x7f970c0ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9710e00640 1 -- 172.21.15.132:0/4003234137 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f40094d0 con 0x7f970c15d480 2024-03-23T07:55:55.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f970be00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c06f0f0 0x7f970c0c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:55.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9710e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f970c15d480 0x7f970c14d390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9704009d90 tx=0x7f9704004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9708c00640 1 -- 172.21.15.132:0/4003234137 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970402cea0 con 0x7f970c15d480 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9708c00640 1 -- 172.21.15.132:0/4003234137 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9704040650 con 0x7f970c15d480 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9708c00640 1 -- 172.21.15.132:0/4003234137 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704040a00 con 0x7f970c15d480 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.767+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9704009a70 con 0x7f970c15d480 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.768+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970c0c5a20 con 0x7f970c15d480 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.768+0000 7f9708c00640 1 -- 172.21.15.132:0/4003234137 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9704049400 con 0x7f970c15d480 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.768+0000 7f9708c00640 1 -- 172.21.15.132:0/4003234137 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f970403e910 con 0x7f970c15d480 2024-03-23T07:55:55.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.769+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970c066740 con 0x7f970c15d480 2024-03-23T07:55:55.770 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:55.770+0000 7fb386753040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T07:55:55.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.771+0000 7f9708c00640 1 -- 172.21.15.132:0/4003234137 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f9704067b50 con 0x7f970c15d480 2024-03-23T07:55:55.795 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:55.795+0000 7f181042e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T07:55:55.823 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:55.822+0000 7fb386753040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T07:55:55.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.846+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d1bd131a-59ed-4ac8-b3ff-cd7074eb8387", "id": 4} v 0) v1 -- 0x7f970c066bb0 con 0x7f970c15d480 2024-03-23T07:55:55.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.856+0000 7f9708c00640 1 -- 172.21.15.132:0/4003234137 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1bd131a-59ed-4ac8-b3ff-cd7074eb8387", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9704056030 con 0x7f970c15d480 2024-03-23T07:55:55.856 INFO:teuthology.orchestra.run.smithi132.stdout:4 2024-03-23T07:55:55.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f970c15d480 msgr2=0x7f970c14d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:55.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f970c15d480 0x7f970c14d390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9704009d90 tx=0x7f9704004a40 comp rx=0 tx=0).stop 2024-03-23T07:55:55.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 shutdown_connections 2024-03-23T07:55:55.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f970c15d480 0x7f970c14d390 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f970c15cb60 0x7f970c0ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 --2- 172.21.15.132:0/4003234137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f970c06f0f0 0x7f970c0c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:55.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 >> 172.21.15.132:0/4003234137 conn(0x7f970c0ced50 msgr2=0x7f970c0cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:55.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 shutdown_connections 2024-03-23T07:55:55.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:55.857+0000 7f9712e00640 1 -- 172.21.15.132:0/4003234137 wait complete. 2024-03-23T07:55:55.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 5f6cdc52-6519-4e96-be59-77624d9dea1a 5 2024-03-23T07:55:55.938 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:55.938+0000 7f181042e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T07:55:55.948 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:55.948+0000 7fb386753040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T07:55:55.994 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:55.994+0000 7f181042e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T07:55:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.054+0000 7fc651600640 1 Processor -- start 2024-03-23T07:55:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.054+0000 7fc651600640 1 -- start start 2024-03-23T07:55:56.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.055+0000 7fc651600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc64c15cd10 0x7fc64c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.055+0000 7fc651600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.055+0000 7fc651600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c1617d0 con 0x7fc64c0b47d0 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.055+0000 7fc651600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c0b5150 con 0x7fc64c15cd10 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.055+0000 7fc651600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c160c50 con 0x7fc64c0d1f30 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc64c15cd10 0x7fc64c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc64c15cd10 0x7fc64c0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45658/0 (socket says 172.21.15.132:45658) 2024-03-23T07:55:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64aa00640 1 -- 172.21.15.132:0/3058907247 learned_addr learned my addr 172.21.15.132:0/3058907247 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:56.057 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:56.056+0000 7fb386753040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64aa00640 1 -- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 msgr2=0x7fc64c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64aa00640 1 --2- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64aa00640 1 -- 172.21.15.132:0/3058907247 >> v1:172.21.15.132:6789/0 conn(0x7fc64c0b47d0 legacy=0x7fc64c06b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64aa00640 1 -- 172.21.15.132:0/3058907247 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c160f30 con 0x7fc64c15cd10 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64a000640 1 -- 172.21.15.132:0/3058907247 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311464509 0 0) 0x7fc64c1617d0 con 0x7fc64c0b47d0 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.056+0000 7fc64b400640 1 --2- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc64aa00640 1 --2- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c0cdc40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009870 tx=0x7fc63c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc766cc3b3e6dd3a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc64a000640 1 -- 172.21.15.132:0/3058907247 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c030840 con 0x7fc64c15cd10 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc64a000640 1 -- 172.21.15.132:0/3058907247 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c0309a0 con 0x7fc64c15cd10 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc64a000640 1 -- 172.21.15.132:0/3058907247 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c030d30 con 0x7fc64c15cd10 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc651600640 1 -- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 msgr2=0x7fc64c0cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc651600640 1 --2- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c0cdc40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009870 tx=0x7fc63c02b980 comp rx=0 tx=0).stop 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc651600640 1 -- 172.21.15.132:0/3058907247 shutdown_connections 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc651600640 1 --2- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc651600640 1 --2- 172.21.15.132:0/3058907247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c0cdc40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.058+0000 7fc651600640 1 -- 172.21.15.132:0/3058907247 >> 172.21.15.132:0/3058907247 conn(0x7fc64c0cf830 msgr2=0x7fc64c0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.059+0000 7fc651600640 1 -- 172.21.15.132:0/3058907247 shutdown_connections 2024-03-23T07:55:56.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.059+0000 7fc651600640 1 -- 172.21.15.132:0/3058907247 wait complete. 2024-03-23T07:55:56.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.060+0000 7fc651600640 1 Processor -- start 2024-03-23T07:55:56.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.060+0000 7fc651600640 1 -- start start 2024-03-23T07:55:56.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.061+0000 7fc651600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc64c0b47d0 0x7fc64c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.061+0000 7fc651600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.061+0000 7fc651600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c1f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.061+0000 7fc651600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c156710 con 0x7fc64c0b47d0 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.061+0000 7fc651600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c15a1e0 con 0x7fc64c15cd10 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.061+0000 7fc651600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0c6d50 con 0x7fc64c0d1f30 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c1f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc643e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c1f1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45668/0 (socket says 172.21.15.132:45668) 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 -- 172.21.15.132:0/908270576 learned_addr learned my addr 172.21.15.132:0/908270576 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64aa00640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc64c0b47d0 0x7fc64c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 -- 172.21.15.132:0/908270576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 msgr2=0x7fc64c0cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 -- 172.21.15.132:0/908270576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc64c0b47d0 msgr2=0x7fc64c0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc64c0b47d0 0x7fc64c0c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 -- 172.21.15.132:0/908270576 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644009a70 con 0x7fc64c15cd10 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64aa00640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc64c0b47d0 0x7fc64c0c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.062+0000 7fc64b400640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c1f1bc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc64400ef90 tx=0x7fc644009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.063+0000 7fc642a00640 1 -- 172.21.15.132:0/908270576 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644009d70 con 0x7fc64c15cd10 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.063+0000 7fc642a00640 1 -- 172.21.15.132:0/908270576 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64400f040 con 0x7fc64c15cd10 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.063+0000 7fc642a00640 1 -- 172.21.15.132:0/908270576 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440146c0 con 0x7fc64c15cd10 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.063+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c0094d0 con 0x7fc64c15cd10 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.063+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c0c7280 con 0x7fc64c15cd10 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.063+0000 7fc642a00640 1 -- 172.21.15.132:0/908270576 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc644014b20 con 0x7fc64c15cd10 2024-03-23T07:55:56.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.063+0000 7fc642a00640 1 -- 172.21.15.132:0/908270576 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fc64401d980 con 0x7fc64c15cd10 2024-03-23T07:55:56.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.064+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c066bb0 con 0x7fc64c15cd10 2024-03-23T07:55:56.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.066+0000 7fc642a00640 1 -- 172.21.15.132:0/908270576 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fc64403cb50 con 0x7fc64c15cd10 2024-03-23T07:55:56.126 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:56.126+0000 7f181042e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T07:55:56.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.143+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5f6cdc52-6519-4e96-be59-77624d9dea1a", "id": 5} v 0) v1 -- 0x7fc64c0b11f0 con 0x7fc64c15cd10 2024-03-23T07:55:56.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.154+0000 7fc642a00640 1 -- 172.21.15.132:0/908270576 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5f6cdc52-6519-4e96-be59-77624d9dea1a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc64402b070 con 0x7fc64c15cd10 2024-03-23T07:55:56.154 INFO:teuthology.orchestra.run.smithi132.stdout:5 2024-03-23T07:55:56.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.155+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 msgr2=0x7fc64c1f1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.155+0000 7fc651600640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c1f1bc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc64400ef90 tx=0x7fc644009f90 comp rx=0 tx=0).stop 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.155+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 shutdown_connections 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.155+0000 7fc651600640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c15cd10 0x7fc64c1f1bc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.155+0000 7fc651600640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c0d1f30 0x7fc64c0cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.155+0000 7fc651600640 1 --2- 172.21.15.132:0/908270576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc64c0b47d0 0x7fc64c0c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.155+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 >> 172.21.15.132:0/908270576 conn(0x7fc64c0cf830 msgr2=0x7fc64c0b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.156+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 shutdown_connections 2024-03-23T07:55:56.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.156+0000 7fc651600640 1 -- 172.21.15.132:0/908270576 wait complete. 2024-03-23T07:55:56.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new bb44da31-9fd8-4442-bdb0-0b1cdae07bf8 6 2024-03-23T07:55:56.232 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:56.231+0000 7fb386753040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T07:55:56.248 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:56.249+0000 7f181042e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T07:55:56.280 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:56.280+0000 7fb386753040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T07:55:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.358+0000 7f95c6000640 1 Processor -- start 2024-03-23T07:55:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.359+0000 7f95c6000640 1 -- start start 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.360+0000 7f95c6000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95c00b84b0 0x7f95c00d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.360+0000 7f95c6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00b5aa0 0x7f95c00b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.360+0000 7f95c6000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00d1280 con 0x7f95c00cd740 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.360+0000 7f95c6000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c00d20c0 con 0x7f95c00b84b0 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.360+0000 7f95c6000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00b8940 con 0x7f95c00b5aa0 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.360+0000 7f95b7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95c00b84b0 0x7f95c00d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.360+0000 7f95bf400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f95c00cd740 0x7f95c00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38608/0 (socket says 172.21.15.132:38608) 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.361+0000 7f95bf400640 1 -- 172.21.15.132:0/324406679 learned_addr learned my addr 172.21.15.132:0/324406679 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.361+0000 7f95bfe00640 1 --2- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00b5aa0 0x7f95c00b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.361+0000 7f95b7e00640 1 -- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00b5aa0 msgr2=0x7f95c00b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.361+0000 7f95b7e00640 1 --2- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00b5aa0 0x7f95c00b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.361+0000 7f95b7e00640 1 -- 172.21.15.132:0/324406679 >> v1:172.21.15.132:6789/0 conn(0x7f95c00cd740 legacy=0x7f95c00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.361+0000 7f95b7e00640 1 -- 172.21.15.132:0/324406679 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00d14b0 con 0x7f95c00b84b0 2024-03-23T07:55:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.362+0000 7f95b7e00640 1 --2- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 0x7f95c00d0d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95a80097a0 tx=0x7f95a802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43a033a086f06f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95bea00640 1 -- 172.21.15.132:0/324406679 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a802d040 con 0x7f95c00b84b0 2024-03-23T07:55:56.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95bea00640 1 -- 172.21.15.132:0/324406679 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8031810 con 0x7f95c00b84b0 2024-03-23T07:55:56.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95bea00640 1 -- 172.21.15.132:0/324406679 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8031ba0 con 0x7f95c00b84b0 2024-03-23T07:55:56.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95c6000640 1 -- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 msgr2=0x7f95c00d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95c6000640 1 --2- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 0x7f95c00d0d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95a80097a0 tx=0x7f95a802b8b0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95c6000640 1 -- 172.21.15.132:0/324406679 shutdown_connections 2024-03-23T07:55:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95c6000640 1 --2- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00b5aa0 0x7f95c00b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95c6000640 1 --2- 172.21.15.132:0/324406679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 0x7f95c00d0d40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.363+0000 7f95c6000640 1 -- 172.21.15.132:0/324406679 >> 172.21.15.132:0/324406679 conn(0x7f95c00cf5d0 msgr2=0x7f95c00cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.364+0000 7f95c6000640 1 -- 172.21.15.132:0/324406679 shutdown_connections 2024-03-23T07:55:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.364+0000 7f95c6000640 1 -- 172.21.15.132:0/324406679 wait complete. 2024-03-23T07:55:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.365+0000 7f95c6000640 1 Processor -- start 2024-03-23T07:55:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.365+0000 7f95c6000640 1 -- start start 2024-03-23T07:55:56.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95c6000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95c00b5aa0 0x7f95c00c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95c6000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 0x7f95c00cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95c6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00cd740 0x7f95c00cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95c6000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00cad30 con 0x7f95c00b5aa0 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95c6000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00cc500 con 0x7f95c00b84b0 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95c6000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c014a640 con 0x7f95c00cd740 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00cd740 0x7f95c00cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95b7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 0x7f95c00cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00cd740 0x7f95c00cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:36302/0 (socket says 172.21.15.132:36302) 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.366+0000 7f95bfe00640 1 -- 172.21.15.132:0/1496377510 learned_addr learned my addr 172.21.15.132:0/1496377510 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bf400640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95c00b5aa0 0x7f95c00c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bfe00640 1 -- 172.21.15.132:0/1496377510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 msgr2=0x7f95c00cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bfe00640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 0x7f95c00cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bfe00640 1 -- 172.21.15.132:0/1496377510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95c00b5aa0 msgr2=0x7f95c00c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bfe00640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95c00b5aa0 0x7f95c00c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bfe00640 1 -- 172.21.15.132:0/1496377510 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8009400 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bf400640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95c00b5aa0 0x7f95c00c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95bfe00640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00cd740 0x7f95c00cbeb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95b800efc0 tx=0x7f95b800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95b7400640 1 -- 172.21.15.132:0/1496377510 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8009ba0 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95b7400640 1 -- 172.21.15.132:0/1496377510 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b800f040 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95b7400640 1 -- 172.21.15.132:0/1496377510 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80146c0 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.367+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c014a920 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.368+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c00cc7e0 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.368+0000 7f95b7400640 1 -- 172.21.15.132:0/1496377510 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f95b8014b20 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.368+0000 7f95b7400640 1 -- 172.21.15.132:0/1496377510 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f95b8014dd0 con 0x7f95c00cd740 2024-03-23T07:55:56.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.369+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c014ae40 con 0x7f95c00cd740 2024-03-23T07:55:56.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.371+0000 7f95b7400640 1 -- 172.21.15.132:0/1496377510 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f95c014ae40 con 0x7f95c00cd740 2024-03-23T07:55:56.434 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:56.434+0000 7f181042e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T07:55:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.448+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bb44da31-9fd8-4442-bdb0-0b1cdae07bf8", "id": 6} v 0) v1 -- 0x7f95c0073390 con 0x7f95c00cd740 2024-03-23T07:55:56.450 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:56.450+0000 7fb386753040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T07:55:56.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.459+0000 7f95b7400640 1 -- 172.21.15.132:0/1496377510 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb44da31-9fd8-4442-bdb0-0b1cdae07bf8", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f95b8018020 con 0x7f95c00cd740 2024-03-23T07:55:56.459 INFO:teuthology.orchestra.run.smithi132.stdout:6 2024-03-23T07:55:56.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00cd740 msgr2=0x7f95c00cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00cd740 0x7f95c00cbeb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95b800efc0 tx=0x7f95b800c490 comp rx=0 tx=0).stop 2024-03-23T07:55:56.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 shutdown_connections 2024-03-23T07:55:56.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95c00cd740 0x7f95c00cbeb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95c00b84b0 0x7f95c00cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 --2- 172.21.15.132:0/1496377510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95c00b5aa0 0x7f95c00c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 >> 172.21.15.132:0/1496377510 conn(0x7f95c00cf5d0 msgr2=0x7f95c015c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:56.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 shutdown_connections 2024-03-23T07:55:56.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.460+0000 7f95c6000640 1 -- 172.21.15.132:0/1496377510 wait complete. 2024-03-23T07:55:56.476 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new a643e4ae-7365-4306-8e2e-1502e320da23 7 2024-03-23T07:55:56.486 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:56.487+0000 7f181042e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T07:55:56.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.664+0000 7f18b9200640 1 Processor -- start 2024-03-23T07:55:56.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.665+0000 7f18b9200640 1 -- start start 2024-03-23T07:55:56.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b9200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f18b415cc10 0x7f18b40afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b40cf400 0x7f18b40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b9200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b40caea0 con 0x7f18b406b7b0 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b9200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b40cb180 con 0x7f18b415cc10 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b9200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40cb460 con 0x7f18b40cf400 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b40cf400 0x7f18b40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b3e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f18b406b7b0 0x7f18b40b10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38622/0 (socket says 172.21.15.132:38622) 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b3e00640 1 -- 172.21.15.132:0/3807431713 learned_addr learned my addr 172.21.15.132:0/3807431713 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b3400640 1 --2- 172.21.15.132:0/3807431713 >> v2:172.21.15.105:3300/0 conn(0x7f18b415cc10 0x7f18b40afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18abe00640 1 -- 172.21.15.132:0/3807431713 >> v2:172.21.15.105:3300/0 conn(0x7f18b415cc10 msgr2=0x7f18b40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18abe00640 1 --2- 172.21.15.132:0/3807431713 >> v2:172.21.15.105:3300/0 conn(0x7f18b415cc10 0x7f18b40afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18abe00640 1 -- 172.21.15.132:0/3807431713 >> v1:172.21.15.132:6789/0 conn(0x7f18b406b7b0 legacy=0x7f18b40b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18abe00640 1 -- 172.21.15.132:0/3807431713 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40cb740 con 0x7f18b40cf400 2024-03-23T07:55:56.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.667+0000 7f18b2a00640 1 -- 172.21.15.132:0/3807431713 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826844772 0 0) 0x7f18b40caea0 con 0x7f18b406b7b0 2024-03-23T07:55:56.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.669+0000 7f18abe00640 1 --2- 172.21.15.132:0/3807431713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b40cf400 0x7f18b40cf840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f189c0097a0 tx=0x7f189c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0abd04b7d439abc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.669+0000 7f18b2a00640 1 -- 172.21.15.132:0/3807431713 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c031830 con 0x7f18b40cf400 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.669+0000 7f18b2a00640 1 -- 172.21.15.132:0/3807431713 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c031990 con 0x7f18b40cf400 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.669+0000 7f18b2a00640 1 -- 172.21.15.132:0/3807431713 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c031d20 con 0x7f18b40cf400 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 -- 172.21.15.132:0/3807431713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b40cf400 msgr2=0x7f18b40cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 --2- 172.21.15.132:0/3807431713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b40cf400 0x7f18b40cf840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f189c0097a0 tx=0x7f189c02b8d0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 -- 172.21.15.132:0/3807431713 shutdown_connections 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 --2- 172.21.15.132:0/3807431713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b40cf400 0x7f18b40cf840 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 --2- 172.21.15.132:0/3807431713 >> v2:172.21.15.105:3300/0 conn(0x7f18b415cc10 0x7f18b40afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 -- 172.21.15.132:0/3807431713 >> 172.21.15.132:0/3807431713 conn(0x7f18b40b2010 msgr2=0x7f18b40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 -- 172.21.15.132:0/3807431713 shutdown_connections 2024-03-23T07:55:56.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f18b9200640 1 -- 172.21.15.132:0/3807431713 wait complete. 2024-03-23T07:55:56.671 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:56.670+0000 7f181042e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T07:55:56.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.672+0000 7f18b9200640 1 Processor -- start 2024-03-23T07:55:56.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.672+0000 7f18b9200640 1 -- start start 2024-03-23T07:55:56.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b406b7b0 0x7f18b40c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b40cf400 0x7f18b415f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b9200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18b4155d10 0x7f18b40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:56.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b9200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b415ff00 con 0x7f18b40cf400 2024-03-23T07:55:56.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b9200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40c7c40 con 0x7f18b4155d10 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b9200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40c7f20 con 0x7f18b406b7b0 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b406b7b0 0x7f18b40c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b406b7b0 0x7f18b40c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:36324/0 (socket says 172.21.15.132:36324) 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 -- 172.21.15.132:0/3805752131 learned_addr learned my addr 172.21.15.132:0/3805752131 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3e00640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18b4155d10 0x7f18b40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18abe00640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b40cf400 0x7f18b415f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 -- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18b4155d10 msgr2=0x7f18b40b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18b4155d10 0x7f18b40b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 -- 172.21.15.132:0/3805752131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b40cf400 msgr2=0x7f18b415f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b40cf400 0x7f18b415f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3400640 1 -- 172.21.15.132:0/3805752131 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c009400 con 0x7f18b406b7b0 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.673+0000 7f18b3e00640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18b4155d10 0x7f18b40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:55:56.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18b3400640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b406b7b0 0x7f18b40c75f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d5f0 tx=0x7f18a400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18ab400640 1 -- 172.21.15.132:0/3805752131 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40125f0 con 0x7f18b406b7b0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18ab400640 1 -- 172.21.15.132:0/3805752131 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a4012750 con 0x7f18b406b7b0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18ab400640 1 -- 172.21.15.132:0/3805752131 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4012b00 con 0x7f18b406b7b0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b40c8200 con 0x7f18b406b7b0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b40c87c0 con 0x7f18b406b7b0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18ab400640 1 -- 172.21.15.132:0/3805752131 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f18a401e460 con 0x7f18b406b7b0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.674+0000 7f18ab400640 1 -- 172.21.15.132:0/3805752131 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f18a4014b30 con 0x7f18b406b7b0 2024-03-23T07:55:56.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.675+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b404e030 con 0x7f18b406b7b0 2024-03-23T07:55:56.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.678+0000 7f18ab400640 1 -- 172.21.15.132:0/3805752131 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f18a4014d80 con 0x7f18b406b7b0 2024-03-23T07:55:56.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.755+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a643e4ae-7365-4306-8e2e-1502e320da23", "id": 7} v 0) v1 -- 0x7f18b40c6fb0 con 0x7f18b406b7b0 2024-03-23T07:55:56.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.767+0000 7f18ab400640 1 -- 172.21.15.132:0/3805752131 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a643e4ae-7365-4306-8e2e-1502e320da23", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f18a40144d0 con 0x7f18b406b7b0 2024-03-23T07:55:56.767 INFO:teuthology.orchestra.run.smithi132.stdout:7 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.768+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b406b7b0 msgr2=0x7f18b40c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.768+0000 7f18b9200640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b406b7b0 0x7f18b40c75f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d5f0 tx=0x7f18a400dab0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.768+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 shutdown_connections 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.768+0000 7f18b9200640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18b4155d10 0x7f18b40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.768+0000 7f18b9200640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18b40cf400 0x7f18b415f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.768+0000 7f18b9200640 1 --2- 172.21.15.132:0/3805752131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18b406b7b0 0x7f18b40c75f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.768+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 >> 172.21.15.132:0/3805752131 conn(0x7f18b40b2010 msgr2=0x7f18b40b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.769+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 shutdown_connections 2024-03-23T07:55:56.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-23T07:55:56.769+0000 7f18b9200640 1 -- 172.21.15.132:0/3805752131 wait complete. 2024-03-23T07:55:56.783 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-23T07:55:56.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-23T07:55:56.790 INFO:tasks.ceph.osd.0:Started 2024-03-23T07:55:56.790 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-23T07:55:56.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-23T07:55:56.796 INFO:tasks.ceph.osd.1:Started 2024-03-23T07:55:56.796 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-23T07:55:56.796 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-23T07:55:56.801 INFO:tasks.ceph.osd.2:Started 2024-03-23T07:55:56.801 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-23T07:55:56.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-23T07:55:56.806 INFO:tasks.ceph.osd.3:Started 2024-03-23T07:55:56.806 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-23T07:55:56.806 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-23T07:55:56.829 INFO:tasks.ceph.osd.4:Started 2024-03-23T07:55:56.829 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-23T07:55:56.829 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-23T07:55:56.836 INFO:tasks.ceph.osd.5:Started 2024-03-23T07:55:56.836 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-23T07:55:56.836 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-23T07:55:56.840 INFO:tasks.ceph.osd.6:Started 2024-03-23T07:55:56.841 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-23T07:55:56.841 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-23T07:55:56.846 INFO:tasks.ceph.osd.7:Started 2024-03-23T07:55:56.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:55:57.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.052+0000 7f6120200640 1 Processor -- start 2024-03-23T07:55:57.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.052+0000 7f6120200640 1 -- start start 2024-03-23T07:55:57.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.053+0000 7f6120200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61180d0af0 0x7f61180d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.053+0000 7f6120200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611815d090 con 0x7f61180b4390 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.053+0000 7f6120200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61180d0f30 con 0x7f611815ccb0 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.053+0000 7f6120200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61180d27d0 con 0x7f61180d0af0 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.054+0000 7f611ce00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61180d0af0 0x7f61180d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.054+0000 7f611ce00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61180d0af0 0x7f61180d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36374/0 (socket says 172.21.15.105:36374) 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.054+0000 7f611ce00640 1 -- 172.21.15.105:0/1199791251 learned_addr learned my addr 172.21.15.105:0/1199791251 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.054+0000 7f611ce00640 1 -- 172.21.15.105:0/1199791251 >> v1:172.21.15.132:6789/0 conn(0x7f611815ccb0 legacy=0x7f61180cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.054+0000 7f611ce00640 1 -- 172.21.15.105:0/1199791251 >> v1:172.21.15.105:6789/0 conn(0x7f61180b4390 legacy=0x7f61180b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.054+0000 7f611ce00640 1 -- 172.21.15.105:0/1199791251 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180d11f0 con 0x7f61180d0af0 2024-03-23T07:55:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.054+0000 7f6117e00640 1 -- 172.21.15.105:0/1199791251 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427848116 0 0) 0x7f61180d0f30 con 0x7f611815ccb0 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.055+0000 7f611ce00640 1 --2- 172.21.15.105:0/1199791251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f61180d2290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6104009870 tx=0x7f610402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571543f98defb28b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.055+0000 7f6117e00640 1 -- 172.21.15.105:0/1199791251 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61040317d0 con 0x7f61180d0af0 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.055+0000 7f6117e00640 1 -- 172.21.15.105:0/1199791251 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6104031930 con 0x7f61180d0af0 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.055+0000 7f6117e00640 1 -- 172.21.15.105:0/1199791251 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104031cc0 con 0x7f61180d0af0 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.056+0000 7f6120200640 1 -- 172.21.15.105:0/1199791251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 msgr2=0x7f61180d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.056+0000 7f6120200640 1 --2- 172.21.15.105:0/1199791251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f61180d2290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6104009870 tx=0x7f610402b8f0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.056+0000 7f6120200640 1 -- 172.21.15.105:0/1199791251 shutdown_connections 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.056+0000 7f6120200640 1 --2- 172.21.15.105:0/1199791251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f61180d2290 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.056+0000 7f6120200640 1 -- 172.21.15.105:0/1199791251 >> 172.21.15.105:0/1199791251 conn(0x7f61180717b0 msgr2=0x7f6118071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.056+0000 7f6120200640 1 -- 172.21.15.105:0/1199791251 shutdown_connections 2024-03-23T07:55:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.056+0000 7f6120200640 1 -- 172.21.15.105:0/1199791251 wait complete. 2024-03-23T07:55:57.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.057+0000 7f6120200640 1 Processor -- start 2024-03-23T07:55:57.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.058+0000 7f6120200640 1 -- start start 2024-03-23T07:55:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.058+0000 7f6120200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180b4390 0x7f61181586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.058+0000 7f6120200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f6118156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.058+0000 7f6120200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f611815ccb0 0x7f61181f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.058+0000 7f6120200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611815e750 con 0x7f61180d0af0 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.058+0000 7f6120200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6118158c20 con 0x7f611815ccb0 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.058+0000 7f6120200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6118158ea0 con 0x7f61180b4390 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f6118156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f6118156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36376/0 (socket says 172.21.15.105:36376) 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 -- 172.21.15.105:0/1050977911 learned_addr learned my addr 172.21.15.105:0/1050977911 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611e200640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f611815ccb0 0x7f61181f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611d800640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180b4390 0x7f61181586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 -- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180b4390 msgr2=0x7f61181586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180b4390 0x7f61181586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 -- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f611815ccb0 msgr2=0x7f61181f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f611815ccb0 0x7f61181f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 -- 172.21.15.105:0/1050977911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61040094d0 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.059+0000 7f611ce00640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f6118156300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6104000c00 tx=0x7f61040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.060+0000 7f6115600640 1 -- 172.21.15.105:0/1050977911 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61040329b0 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.060+0000 7f6115600640 1 -- 172.21.15.105:0/1050977911 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6104032b30 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.060+0000 7f6115600640 1 -- 172.21.15.105:0/1050977911 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61040453f0 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.060+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6118159180 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.060+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611815e370 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.060+0000 7f6115600640 1 -- 172.21.15.105:0/1050977911 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6104030070 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.060+0000 7f6115600640 1 -- 172.21.15.105:0/1050977911 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f61040065c0 con 0x7f61180d0af0 2024-03-23T07:55:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.061+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611815e890 con 0x7f61180d0af0 2024-03-23T07:55:57.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.063+0000 7f6115600640 1 -- 172.21.15.105:0/1050977911 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f611815e890 con 0x7f61180d0af0 2024-03-23T07:55:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.131+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6118066bb0 con 0x7f61180d0af0 2024-03-23T07:55:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.132+0000 7f6115600640 1 -- 172.21.15.105:0/1050977911 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f61040067d0 con 0x7f61180d0af0 2024-03-23T07:55:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:55:57.132 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:55:56.757504+0000","last_up_change":"0.000000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:55:57.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 msgr2=0x7f6118156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:57.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f6118156300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6104000c00 tx=0x7f61040046c0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 shutdown_connections 2024-03-23T07:55:57.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f611815ccb0 0x7f61181f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180d0af0 0x7f6118156300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 --2- 172.21.15.105:0/1050977911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180b4390 0x7f61181586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:57.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 >> 172.21.15.105:0/1050977911 conn(0x7f61180717b0 msgr2=0x7f61180726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:57.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 shutdown_connections 2024-03-23T07:55:57.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:55:57.133+0000 7f6120200640 1 -- 172.21.15.105:0/1050977911 wait complete. 2024-03-23T07:55:57.149 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-23T07:55:57.150 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-23T07:55:57.489 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-23T07:55:57.488+0000 7fbed90104c0 -1 Falling back to public interface 2024-03-23T07:55:57.510 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-23T07:55:57.510+0000 7f7a3c6214c0 -1 Falling back to public interface 2024-03-23T07:55:57.514 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-23T07:55:57.514+0000 7fd4e48184c0 -1 Falling back to public interface 2024-03-23T07:55:57.516 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-03-23T07:55:57.516+0000 7f88ca3224c0 -1 Falling back to public interface 2024-03-23T07:55:57.517 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:57.517+0000 7fb386753040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T07:55:57.521 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-03-23T07:55:57.521+0000 7f974c28c4c0 -1 Falling back to public interface 2024-03-23T07:55:57.536 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-23T07:55:57.535+0000 7f4f426a74c0 -1 Falling back to public interface 2024-03-23T07:55:57.557 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-03-23T07:55:57.557+0000 7f1b9ca9e4c0 -1 Falling back to public interface 2024-03-23T07:55:57.557 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-03-23T07:55:57.557+0000 7fd5eb8204c0 -1 Falling back to public interface 2024-03-23T07:55:57.816 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:57.816+0000 7f181042e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T07:55:57.827 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:57.826+0000 7fb386753040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T07:55:57.886 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:57.885+0000 7fb386753040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T07:55:57.955 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T07:55:58.018 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:58.017+0000 7fb386753040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T07:55:58.119 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.118+0000 7f8a1b200640 1 Processor -- start 2024-03-23T07:55:58.119 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.118+0000 7f8a1b200640 1 -- start start 2024-03-23T07:55:58.120 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a1b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a140b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:58.120 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a1b200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a140cd8c0 0x7f8a140b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:58.121 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a1b200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1415d210 con 0x7f8a1415ce30 2024-03-23T07:55:58.121 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a1b200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140b2180 con 0x7f8a140cd8c0 2024-03-23T07:55:58.121 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a1b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140b88e0 con 0x7f8a140b4af0 2024-03-23T07:55:58.121 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a13e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8a1415ce30 0x7f8a140b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37274/0 (socket says 172.21.15.105:37274) 2024-03-23T07:55:58.121 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a13e00640 1 -- 172.21.15.105:0/983922564 learned_addr learned my addr 172.21.15.105:0/983922564 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:55:58.121 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a13400640 1 --2- 172.21.15.105:0/983922564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a140b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:58.121 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a19200640 1 --2- 172.21.15.105:0/983922564 >> v2:172.21.15.105:3300/0 conn(0x7f8a140cd8c0 0x7f8a140b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:58.122 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a19200640 1 -- 172.21.15.105:0/983922564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 msgr2=0x7f8a140b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:55:58.122 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a19200640 1 --2- 172.21.15.105:0/983922564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a140b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.122 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a19200640 1 -- 172.21.15.105:0/983922564 >> v1:172.21.15.105:6789/0 conn(0x7f8a1415ce30 legacy=0x7f8a140b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:58.122 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a19200640 1 -- 172.21.15.105:0/983922564 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140cddf0 con 0x7f8a140cd8c0 2024-03-23T07:55:58.122 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.120+0000 7f8a12a00640 1 -- 172.21.15.105:0/983922564 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053822799 0 0) 0x7f8a04001490 con 0x7f8a1415ce30 2024-03-23T07:55:58.122 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a19200640 1 --2- 172.21.15.105:0/983922564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140b5a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0097a0 tx=0x7f8a0c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5ed39759b374e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:58.123 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a12a00640 1 -- 172.21.15.105:0/983922564 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c030800 con 0x7f8a140cd8c0 2024-03-23T07:55:58.123 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a12a00640 1 -- 172.21.15.105:0/983922564 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c030960 con 0x7f8a140cd8c0 2024-03-23T07:55:58.123 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a12a00640 1 -- 172.21.15.105:0/983922564 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c030cf0 con 0x7f8a140cd8c0 2024-03-23T07:55:58.123 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a1b200640 1 -- 172.21.15.105:0/983922564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 msgr2=0x7f8a140b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a1b200640 1 --2- 172.21.15.105:0/983922564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140b5a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0097a0 tx=0x7f8a0c02b940 comp rx=0 tx=0).stop 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a1b200640 1 -- 172.21.15.105:0/983922564 shutdown_connections 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a1b200640 1 --2- 172.21.15.105:0/983922564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140b5a90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a1b200640 1 --2- 172.21.15.105:0/983922564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a140b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.122+0000 7f8a1b200640 1 -- 172.21.15.105:0/983922564 >> 172.21.15.105:0/983922564 conn(0x7f8a140cf3e0 msgr2=0x7f8a140cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.123+0000 7f8a1b200640 1 -- 172.21.15.105:0/983922564 shutdown_connections 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.123+0000 7f8a1b200640 1 -- 172.21.15.105:0/983922564 wait complete. 2024-03-23T07:55:58.124 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.124+0000 7f8a1b200640 1 Processor -- start 2024-03-23T07:55:58.125 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.124+0000 7f8a1b200640 1 -- start start 2024-03-23T07:55:58.127 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.125+0000 7f8a1b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a14158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:58.127 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.125+0000 7f8a1b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:58.127 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.125+0000 7f8a1b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a1415ce30 0x7f8a1415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:55:58.127 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.125+0000 7f8a1b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14159150 con 0x7f8a140cd8c0 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.125+0000 7f8a1b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141f2a40 con 0x7f8a1415ce30 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.125+0000 7f8a1b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141f2d20 con 0x7f8a140b4af0 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36446/0 (socket says 172.21.15.105:36446) 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 -- 172.21.15.105:0/971846448 learned_addr learned my addr 172.21.15.105:0/971846448 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13e00640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a14158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a19200640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a1415ce30 0x7f8a1415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 -- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 msgr2=0x7f8a14158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a14158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 -- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a1415ce30 msgr2=0x7f8a1415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a1415ce30 0x7f8a1415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 -- 172.21.15.105:0/971846448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c009400 con 0x7f8a140cd8c0 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.126+0000 7f8a13400640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140c8ce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9b0 tx=0x7f8a0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:55:58.128 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.127+0000 7f8a0be00640 1 -- 172.21.15.105:0/971846448 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00009670 con 0x7f8a140cd8c0 2024-03-23T07:55:58.129 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.127+0000 7f8a0be00640 1 -- 172.21.15.105:0/971846448 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0000c9a0 con 0x7f8a140cd8c0 2024-03-23T07:55:58.129 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.127+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a141f3000 con 0x7f8a140cd8c0 2024-03-23T07:55:58.129 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.127+0000 7f8a0be00640 1 -- 172.21.15.105:0/971846448 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000cd50 con 0x7f8a140cd8c0 2024-03-23T07:55:58.129 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.127+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a140c9220 con 0x7f8a140cd8c0 2024-03-23T07:55:58.129 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.128+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140670d0 con 0x7f8a140cd8c0 2024-03-23T07:55:58.130 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.128+0000 7f8a0be00640 1 -- 172.21.15.105:0/971846448 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 2) v1 ==== 44520+0+0 (secure 0 0 0) 0x7f8a00013470 con 0x7f8a140cd8c0 2024-03-23T07:55:58.130 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.128+0000 7f8a0be00640 1 -- 172.21.15.105:0/971846448 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8a00049650 con 0x7f8a140cd8c0 2024-03-23T07:55:58.131 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.130+0000 7f8a0be00640 1 -- 172.21.15.105:0/971846448 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f8a000232f0 con 0x7f8a140cd8c0 2024-03-23T07:55:58.156 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:58.156+0000 7f181042e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T07:55:58.195 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.195+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a141f3520 con 0x7f8a140cd8c0 2024-03-23T07:55:58.197 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.196+0000 7f8a0be00640 1 -- 172.21.15.105:0/971846448 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8a00017090 con 0x7f8a140cd8c0 2024-03-23T07:55:58.197 INFO:teuthology.misc.health.smithi105.stdout: 2024-03-23T07:55:58.197 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":10,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:55:56.757504+0000","last_up_change":"0.000000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:55:58.198 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 msgr2=0x7f8a140c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:55:58.198 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140c8ce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9b0 tx=0x7f8a0000ee70 comp rx=0 tx=0).stop 2024-03-23T07:55:58.198 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 shutdown_connections 2024-03-23T07:55:58.198 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a1415ce30 0x7f8a1415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.198 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a140cd8c0 0x7f8a140c8ce0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.198 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 --2- 172.21.15.105:0/971846448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a140b4af0 0x7f8a14158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:55:58.199 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 >> 172.21.15.105:0/971846448 conn(0x7f8a140cf3e0 msgr2=0x7f8a140b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:55:58.199 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 shutdown_connections 2024-03-23T07:55:58.199 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:55:58.197+0000 7f8a1b200640 1 -- 172.21.15.105:0/971846448 wait complete. 2024-03-23T07:55:58.210 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-23T07:55:58.221 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:58.221+0000 7f181042e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T07:55:58.351 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-03-23T07:55:58.351+0000 7f181042e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T07:55:59.061 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:59.060+0000 7fb327a00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-23T07:55:59.061 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-23T07:55:59.060+0000 7fb327a00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-23T07:56:00.866 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-03-23T07:56:00.866+0000 7fd5eb8204c0 -1 osd.5 0 log_to_monitors true 2024-03-23T07:56:01.050 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-23T07:56:01.050+0000 7f7a3c6214c0 -1 osd.2 0 log_to_monitors true 2024-03-23T07:56:01.070 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-23T07:56:01.070+0000 7fd4e48184c0 -1 osd.1 0 log_to_monitors true 2024-03-23T07:56:01.071 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-03-23T07:56:01.071+0000 7f88ca3224c0 -1 osd.4 0 log_to_monitors true 2024-03-23T07:56:01.084 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-03-23T07:56:01.084+0000 7f974c28c4c0 -1 osd.6 0 log_to_monitors true 2024-03-23T07:56:01.122 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-03-23T07:56:01.122+0000 7f1b9ca9e4c0 -1 osd.7 0 log_to_monitors true 2024-03-23T07:56:01.264 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-23T07:56:01.263+0000 7fbed90104c0 -1 osd.0 0 log_to_monitors true 2024-03-23T07:56:01.327 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-23T07:56:01.327+0000 7f4f426a74c0 -1 osd.3 0 log_to_monitors true 2024-03-23T07:56:03.153 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-03-23T07:56:03.153+0000 7fd5e6c00640 -1 osd.5 0 waiting for initial osdmap 2024-03-23T07:56:03.173 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-03-23T07:56:03.173+0000 7fd5dc200640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.016 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:05.118 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-03-23T07:56:05.119+0000 7f9747200640 -1 osd.6 0 waiting for initial osdmap 2024-03-23T07:56:05.124 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-03-23T07:56:05.125+0000 7f88c5200640 -1 osd.4 0 waiting for initial osdmap 2024-03-23T07:56:05.126 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-03-23T07:56:05.126+0000 7f973d200640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.130 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-03-23T07:56:05.130+0000 7f1b97a00640 -1 osd.7 0 waiting for initial osdmap 2024-03-23T07:56:05.131 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-03-23T07:56:05.131+0000 7f88bb200640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.140 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-03-23T07:56:05.140+0000 7f1b8da00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.262 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.261+0000 7fae1f600640 1 Processor -- start 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.262+0000 7fae1f600640 1 -- start start 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.263+0000 7fae1f600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae180b4af0 0x7fae180b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.263+0000 7fae1f600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae180cd8c0 0x7fae180b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.263+0000 7fae1f600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1815d210 con 0x7fae1815ce30 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.263+0000 7fae1f600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae180b2180 con 0x7fae180cd8c0 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.263+0000 7fae1f600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180b88e0 con 0x7fae180b4af0 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.264+0000 7fae17e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae180b4af0 0x7fae180b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:05.266 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.264+0000 7fae1d600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae180cd8c0 0x7fae180b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:05.267 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.264+0000 7fae1d600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae180cd8c0 0x7fae180b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:41474/0 (socket says 172.21.15.105:41474) 2024-03-23T07:56:05.267 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.264+0000 7fae1d600640 1 -- 172.21.15.105:0/3819851316 learned_addr learned my addr 172.21.15.105:0/3819851316 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:05.267 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.265+0000 7fae17e00640 1 -- 172.21.15.105:0/3819851316 >> v2:172.21.15.132:3300/0 conn(0x7fae180cd8c0 msgr2=0x7fae180b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:05.267 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.265+0000 7fae17e00640 1 --2- 172.21.15.105:0/3819851316 >> v2:172.21.15.132:3300/0 conn(0x7fae180cd8c0 0x7fae180b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.267 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.265+0000 7fae17e00640 1 -- 172.21.15.105:0/3819851316 >> v1:172.21.15.105:6789/0 conn(0x7fae1815ce30 legacy=0x7fae180b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:05.267 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.265+0000 7fae17e00640 1 -- 172.21.15.105:0/3819851316 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180cddf0 con 0x7fae180b4af0 2024-03-23T07:56:05.268 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.266+0000 7fae17e00640 1 --2- 172.21.15.105:0/3819851316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae180b4af0 0x7fae180b83a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fae040097a0 tx=0x7fae0402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfaf98d9841340c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:05.268 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.267+0000 7fae17400640 1 -- 172.21.15.105:0/3819851316 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0402d040 con 0x7fae180b4af0 2024-03-23T07:56:05.270 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.269+0000 7fae17400640 1 -- 172.21.15.105:0/3819851316 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae04031810 con 0x7fae180b4af0 2024-03-23T07:56:05.270 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.269+0000 7fae17400640 1 -- 172.21.15.105:0/3819851316 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04031ba0 con 0x7fae180b4af0 2024-03-23T07:56:05.270 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.269+0000 7fae1f600640 1 -- 172.21.15.105:0/3819851316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae180b4af0 msgr2=0x7fae180b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:05.270 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.269+0000 7fae1f600640 1 --2- 172.21.15.105:0/3819851316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae180b4af0 0x7fae180b83a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fae040097a0 tx=0x7fae0402b8b0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.270 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.270+0000 7fae1f600640 1 -- 172.21.15.105:0/3819851316 shutdown_connections 2024-03-23T07:56:05.270 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.270+0000 7fae1f600640 1 --2- 172.21.15.105:0/3819851316 >> v2:172.21.15.132:3300/0 conn(0x7fae180cd8c0 0x7fae180b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.271 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.270+0000 7fae1f600640 1 --2- 172.21.15.105:0/3819851316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae180b4af0 0x7fae180b83a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.271 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.270+0000 7fae1f600640 1 -- 172.21.15.105:0/3819851316 >> 172.21.15.105:0/3819851316 conn(0x7fae180cf3e0 msgr2=0x7fae180cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:05.271 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.270+0000 7fae1f600640 1 -- 172.21.15.105:0/3819851316 shutdown_connections 2024-03-23T07:56:05.271 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.270+0000 7fae1f600640 1 -- 172.21.15.105:0/3819851316 wait complete. 2024-03-23T07:56:05.273 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.272+0000 7fae1f600640 1 Processor -- start 2024-03-23T07:56:05.273 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.272+0000 7fae1f600640 1 -- start start 2024-03-23T07:56:05.274 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.273+0000 7fae1f600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae180b4af0 0x7fae180c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:05.274 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.273+0000 7fae1f600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae180cd8c0 0x7fae1815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:05.274 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.273+0000 7fae1f600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae1815ce30 0x7fae18155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:05.274 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.273+0000 7fae1f600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c76b0 con 0x7fae180b4af0 2024-03-23T07:56:05.274 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.273+0000 7fae1f600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c7990 con 0x7fae180cd8c0 2024-03-23T07:56:05.274 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.273+0000 7fae1f600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c7c70 con 0x7fae1815ce30 2024-03-23T07:56:05.276 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.274+0000 7fae17e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae180cd8c0 0x7fae1815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:05.276 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.274+0000 7fae17e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae180cd8c0 0x7fae1815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:41484/0 (socket says 172.21.15.105:41484) 2024-03-23T07:56:05.276 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.274+0000 7fae17e00640 1 -- 172.21.15.105:0/207201508 learned_addr learned my addr 172.21.15.105:0/207201508 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:05.276 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.274+0000 7fae1d600640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae1815ce30 0x7fae18155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:05.276 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.275+0000 7fae17e00640 1 -- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae1815ce30 msgr2=0x7fae18155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:05.276 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.275+0000 7fae17e00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae1815ce30 0x7fae18155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.276 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.275+0000 7fae17e00640 1 -- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae180b4af0 msgr2=0x7fae180c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:05.277 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.276+0000 7fae1cc00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae180b4af0 0x7fae180c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:05.278 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.277+0000 7fae17e00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae180b4af0 0x7fae180c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.279 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.278+0000 7fae17e00640 1 -- 172.21.15.105:0/207201508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae04009400 con 0x7fae180cd8c0 2024-03-23T07:56:05.279 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.279+0000 7fae1cc00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae180b4af0 0x7fae180c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:05.282 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.281+0000 7fae17e00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae180cd8c0 0x7fae1815afb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae04000c00 tx=0x7fae040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:05.284 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.282+0000 7fae14c00640 1 -- 172.21.15.105:0/207201508 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0403ad60 con 0x7fae180cd8c0 2024-03-23T07:56:05.284 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.283+0000 7fae14c00640 1 -- 172.21.15.105:0/207201508 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae040443f0 con 0x7fae180cd8c0 2024-03-23T07:56:05.284 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.283+0000 7fae14c00640 1 -- 172.21.15.105:0/207201508 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040447a0 con 0x7fae180cd8c0 2024-03-23T07:56:05.284 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.283+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae180c7f50 con 0x7fae180cd8c0 2024-03-23T07:56:05.286 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.284+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1815b4f0 con 0x7fae180cd8c0 2024-03-23T07:56:05.286 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.285+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae180670d0 con 0x7fae180cd8c0 2024-03-23T07:56:05.295 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.289+0000 7fae14c00640 1 -- 172.21.15.105:0/207201508 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7fae0404d450 con 0x7fae180cd8c0 2024-03-23T07:56:05.295 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.295+0000 7fae14c00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadf806ac40 0x7fadf806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:05.296 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.296+0000 7fae14c00640 1 -- 172.21.15.105:0/207201508 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7fae040aab50 con 0x7fae180cd8c0 2024-03-23T07:56:05.299 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.299+0000 7fae1cc00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadf806ac40 0x7fadf806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:05.304 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.303+0000 7fae14c00640 1 -- 172.21.15.105:0/207201508 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fae04076b90 con 0x7fae180cd8c0 2024-03-23T07:56:05.306 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.306+0000 7fae1cc00640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadf806ac40 0x7fadf806d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00a810 tx=0x7fae0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:05.373 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-23T07:56:05.373+0000 7f7a36e00640 -1 osd.2 0 waiting for initial osdmap 2024-03-23T07:56:05.382 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-23T07:56:05.381+0000 7f7a2ce00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.402 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-23T07:56:05.401+0000 7fbed3800640 -1 osd.0 0 waiting for initial osdmap 2024-03-23T07:56:05.409 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-23T07:56:05.408+0000 7fbec9800640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.547 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-23T07:56:05.546+0000 7f4f3d600640 -1 osd.3 0 waiting for initial osdmap 2024-03-23T07:56:05.561 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-23T07:56:05.561+0000 7f4f33600640 -1 osd.3 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.568 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-23T07:56:05.568+0000 7fd4df200640 -1 osd.1 0 waiting for initial osdmap 2024-03-23T07:56:05.587 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-23T07:56:05.587+0000 7fd4d5200640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T07:56:05.614 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.614+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae181ee7c0 con 0x7fae180cd8c0 2024-03-23T07:56:05.616 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.616+0000 7fae14c00640 1 -- 172.21.15.105:0/207201508 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8807 (secure 0 0 0) 0x7fae181ee7c0 con 0x7fae180cd8c0 2024-03-23T07:56:05.616 INFO:teuthology.misc.health.smithi105.stdout: 2024-03-23T07:56:05.617 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":15,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:05.380787+0000","last_up_change":"2024-03-23T07:56:05.380787+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:05.619 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.618+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadf806ac40 msgr2=0x7fadf806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:05.619 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadf806ac40 0x7fadf806d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00a810 tx=0x7fae0c008040 comp rx=0 tx=0).stop 2024-03-23T07:56:05.619 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae180cd8c0 msgr2=0x7fae1815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:05.619 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae180cd8c0 0x7fae1815afb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae04000c00 tx=0x7fae040046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 shutdown_connections 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadf806ac40 0x7fadf806d100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae1815ce30 0x7fae18155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae180cd8c0 0x7fae1815afb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 --2- 172.21.15.105:0/207201508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae180b4af0 0x7fae180c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 >> 172.21.15.105:0/207201508 conn(0x7fae180cf3e0 msgr2=0x7fae180b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 shutdown_connections 2024-03-23T07:56:05.620 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:05.619+0000 7fae1f600640 1 -- 172.21.15.105:0/207201508 wait complete. 2024-03-23T07:56:05.632 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-03-23T07:56:07.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:56:07.746 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:56:12.440 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:12.621 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.622+0000 7f9274200640 1 Processor -- start 2024-03-23T07:56:12.622 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.622+0000 7f9274200640 1 -- start start 2024-03-23T07:56:12.623 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9274200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f926c146380 0x7f926c160550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:12.623 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9274200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c06a060 0x7f926c06a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:12.624 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9274200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f926c06d640 0x7f926c06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:12.624 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9274200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c06aa00 con 0x7f926c146380 2024-03-23T07:56:12.624 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9274200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c06e010 con 0x7f926c06d640 2024-03-23T07:56:12.624 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9274200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0cbfb0 con 0x7f926c06a060 2024-03-23T07:56:12.624 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9271800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f926c146380 0x7f926c160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:12.624 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9271800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f926c146380 0x7f926c160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56860/0 (socket says 172.21.15.105:56860) 2024-03-23T07:56:12.624 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9271800640 1 -- 172.21.15.105:0/1144925927 learned_addr learned my addr 172.21.15.105:0/1144925927 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9272200640 1 --2- 172.21.15.105:0/1144925927 >> v2:172.21.15.132:3300/0 conn(0x7f926c06d640 0x7f926c06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9270e00640 1 --2- 172.21.15.105:0/1144925927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c06a060 0x7f926c06a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9271800640 1 -- 172.21.15.105:0/1144925927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c06a060 msgr2=0x7f926c06a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9271800640 1 --2- 172.21.15.105:0/1144925927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c06a060 0x7f926c06a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9271800640 1 -- 172.21.15.105:0/1144925927 >> v2:172.21.15.132:3300/0 conn(0x7f926c06d640 msgr2=0x7f926c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.624+0000 7f9271800640 1 --2- 172.21.15.105:0/1144925927 >> v2:172.21.15.132:3300/0 conn(0x7f926c06d640 0x7f926c06dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.625+0000 7f9271800640 1 -- 172.21.15.105:0/1144925927 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0cb130 con 0x7f926c146380 2024-03-23T07:56:12.625 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.625+0000 7f9272200640 1 --2- 172.21.15.105:0/1144925927 >> v2:172.21.15.132:3300/0 conn(0x7f926c06d640 0x7f926c06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:12.626 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.626+0000 7f9271800640 1 --2- 172.21.15.105:0/1144925927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c146380 0x7f926c160550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f926c0aff30 tx=0x7f926002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0cdd4d8ef84aa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:12.626 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.626+0000 7f926be00640 1 -- 172.21.15.105:0/1144925927 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600317e0 con 0x7f926c146380 2024-03-23T07:56:12.626 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.626+0000 7f926be00640 1 -- 172.21.15.105:0/1144925927 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260031940 con 0x7f926c146380 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.626+0000 7f926be00640 1 -- 172.21.15.105:0/1144925927 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260031cd0 con 0x7f926c146380 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 -- 172.21.15.105:0/1144925927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c146380 msgr2=0x7f926c160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 --2- 172.21.15.105:0/1144925927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c146380 0x7f926c160550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f926c0aff30 tx=0x7f926002b880 comp rx=0 tx=0).stop 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 -- 172.21.15.105:0/1144925927 shutdown_connections 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 --2- 172.21.15.105:0/1144925927 >> v2:172.21.15.132:3300/0 conn(0x7f926c06d640 0x7f926c06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 --2- 172.21.15.105:0/1144925927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c06a060 0x7f926c06a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 --2- 172.21.15.105:0/1144925927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c146380 0x7f926c160550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 -- 172.21.15.105:0/1144925927 >> 172.21.15.105:0/1144925927 conn(0x7f926c069c80 msgr2=0x7f926c0b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 -- 172.21.15.105:0/1144925927 shutdown_connections 2024-03-23T07:56:12.627 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.627+0000 7f9274200640 1 -- 172.21.15.105:0/1144925927 wait complete. 2024-03-23T07:56:12.628 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.629+0000 7f9274200640 1 Processor -- start 2024-03-23T07:56:12.629 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.629+0000 7f9274200640 1 -- start start 2024-03-23T07:56:12.631 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.630+0000 7f9274200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c06a060 0x7f926c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:12.631 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.630+0000 7f9274200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f926c06d640 0x7f926c132ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:12.631 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.630+0000 7f9274200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c146380 0x7f926c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:12.631 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.630+0000 7f9274200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0b0520 con 0x7f926c06a060 2024-03-23T07:56:12.631 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.630+0000 7f9274200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0ca130 con 0x7f926c06d640 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.630+0000 7f9274200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c8ae0 con 0x7f926c146380 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c06a060 0x7f926c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c06a060 0x7f926c0c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56872/0 (socket says 172.21.15.105:56872) 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 -- 172.21.15.105:0/3492242771 learned_addr learned my addr 172.21.15.105:0/3492242771 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9270e00640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f926c06d640 0x7f926c132ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 -- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c146380 msgr2=0x7f926c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c146380 0x7f926c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 -- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f926c06d640 msgr2=0x7f926c132ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f926c06d640 0x7f926c132ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 -- 172.21.15.105:0/3492242771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600094d0 con 0x7f926c06a060 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9270e00640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f926c06d640 0x7f926c132ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.631+0000 7f9271800640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c06a060 0x7f926c0c9bf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f926002b8c0 tx=0x7f92600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.632+0000 7f9269600640 1 -- 172.21.15.105:0/3492242771 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600329c0 con 0x7f926c06a060 2024-03-23T07:56:12.632 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.632+0000 7f9269600640 1 -- 172.21.15.105:0/3492242771 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260032b40 con 0x7f926c06a060 2024-03-23T07:56:12.633 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.632+0000 7f9269600640 1 -- 172.21.15.105:0/3492242771 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600453f0 con 0x7f926c06a060 2024-03-23T07:56:12.633 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.632+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c0c8dc0 con 0x7f926c06a060 2024-03-23T07:56:12.633 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.632+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c0c9380 con 0x7f926c06a060 2024-03-23T07:56:12.633 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.633+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c071c00 con 0x7f926c06a060 2024-03-23T07:56:12.633 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.634+0000 7f9269600640 1 -- 172.21.15.105:0/3492242771 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f926004d3f0 con 0x7f926c06a060 2024-03-23T07:56:12.634 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.635+0000 7f9269600640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f924c06ac00 0x7f924c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:12.635 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.635+0000 7f9269600640 1 -- 172.21.15.105:0/3492242771 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f92600aaf90 con 0x7f926c06a060 2024-03-23T07:56:12.635 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.635+0000 7f9270e00640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f924c06ac00 0x7f924c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:12.635 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.636+0000 7f9270e00640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f924c06ac00 0x7f924c06d0c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9258006fd0 tx=0x7f9258008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:12.640 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.640+0000 7f9269600640 1 -- 172.21.15.105:0/3492242771 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f92600764d0 con 0x7f926c06a060 2024-03-23T07:56:12.837 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.837+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f926c04e110 con 0x7f926c06a060 2024-03-23T07:56:12.838 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.838+0000 7f9269600640 1 -- 172.21.15.105:0/3492242771 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f9260079af0 con 0x7f926c06a060 2024-03-23T07:56:12.838 INFO:teuthology.misc.health.smithi105.stdout: 2024-03-23T07:56:12.838 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":18,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:08.391996+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f924c06ac00 msgr2=0x7f924c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f924c06ac00 0x7f924c06d0c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9258006fd0 tx=0x7f9258008040 comp rx=0 tx=0).stop 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c06a060 msgr2=0x7f926c0c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c06a060 0x7f926c0c9bf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f926002b8c0 tx=0x7f92600046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 shutdown_connections 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f924c06ac00 0x7f924c06d0c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f926c146380 0x7f926c0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f926c06d640 0x7f926c132ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.841 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 --2- 172.21.15.105:0/3492242771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926c06a060 0x7f926c0c9bf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:12.842 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 >> 172.21.15.105:0/3492242771 conn(0x7f926c069c80 msgr2=0x7f926c070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:12.842 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.841+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 shutdown_connections 2024-03-23T07:56:12.842 INFO:teuthology.misc.health.smithi105.stderr:2024-03-23T07:56:12.842+0000 7f9274200640 1 -- 172.21.15.105:0/3492242771 wait complete. 2024-03-23T07:56:12.855 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-23T07:56:12.855 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-23T07:56:12.856 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T07:56:12.856 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T07:56:12.863 INFO:tasks.ceph.mds.a:Started 2024-03-23T07:56:12.863 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T07:56:12.863 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T07:56:12.870 INFO:tasks.ceph.mds.c:Started 2024-03-23T07:56:12.870 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T07:56:12.870 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T07:56:12.878 INFO:tasks.ceph.mds.b:Started 2024-03-23T07:56:12.878 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T07:56:12.878 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T07:56:12.885 INFO:tasks.ceph.mds.d:Started 2024-03-23T07:56:12.885 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-23T07:56:12.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:13.015 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-03-23T07:56:13.015 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-03-23T07:56:13.039 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-03-23T07:56:13.040 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-03-23T07:56:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.073+0000 7fb378c00640 1 Processor -- start 2024-03-23T07:56:13.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.073+0000 7fb378c00640 1 -- start start 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb378c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b5a30 0x7fb3740b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb378c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb3740b4350 0x7fb3740b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb378c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3740d1830 con 0x7fb37415cda0 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb378c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3740b4d20 con 0x7fb3740b4350 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb378c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3740b1f20 con 0x7fb3740b5a30 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb371e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb37415cda0 0x7fb3740cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.105:50474/0 (socket says 172.21.15.105:50474) 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb371e00640 1 -- 172.21.15.105:0/132579938 learned_addr learned my addr 172.21.15.105:0/132579938 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb372800640 1 --2- 172.21.15.105:0/132579938 >> v2:172.21.15.132:3300/0 conn(0x7fb3740b4350 0x7fb3740b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.074+0000 7fb369400640 1 --2- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b5a30 0x7fb3740b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.075+0000 7fb371400640 1 -- 172.21.15.105:0/132579938 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920037044 0 0) 0x7fb3740d1830 con 0x7fb37415cda0 2024-03-23T07:56:13.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.075+0000 7fb371400640 1 -- 172.21.15.105:0/132579938 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb358003520 con 0x7fb37415cda0 2024-03-23T07:56:13.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.075+0000 7fb372800640 1 -- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b5a30 msgr2=0x7fb3740b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:13.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.075+0000 7fb372800640 1 --2- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b5a30 0x7fb3740b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.075+0000 7fb372800640 1 -- 172.21.15.105:0/132579938 >> v1:172.21.15.132:6789/0 conn(0x7fb37415cda0 legacy=0x7fb3740cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.075+0000 7fb372800640 1 -- 172.21.15.105:0/132579938 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3740d0c20 con 0x7fb3740b4350 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.076+0000 7fb372800640 1 --2- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3740b4350 0x7fb3740b47e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009d90 tx=0x7fb36c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e91d9d398a07a060 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.076+0000 7fb371400640 1 -- 172.21.15.105:0/132579938 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c02cc60 con 0x7fb3740b4350 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.076+0000 7fb371400640 1 -- 172.21.15.105:0/132579938 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36c02cdc0 con 0x7fb3740b4350 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.076+0000 7fb371400640 1 -- 172.21.15.105:0/132579938 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c034660 con 0x7fb3740b4350 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 -- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3740b4350 msgr2=0x7fb3740b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 --2- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3740b4350 0x7fb3740b47e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009d90 tx=0x7fb36c02beb0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 -- 172.21.15.105:0/132579938 shutdown_connections 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 --2- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3740b4350 0x7fb3740b47e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 --2- 172.21.15.105:0/132579938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b5a30 0x7fb3740b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 -- 172.21.15.105:0/132579938 >> 172.21.15.105:0/132579938 conn(0x7fb3740cf350 msgr2=0x7fb3740cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 -- 172.21.15.105:0/132579938 shutdown_connections 2024-03-23T07:56:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.077+0000 7fb378c00640 1 -- 172.21.15.105:0/132579938 wait complete. 2024-03-23T07:56:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.078+0000 7fb378c00640 1 Processor -- start 2024-03-23T07:56:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.079+0000 7fb378c00640 1 -- start start 2024-03-23T07:56:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.079+0000 7fb378c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b4350 0x7fb3741f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.079+0000 7fb378c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3740b5a30 0x7fb3740b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.079+0000 7fb378c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb37415cda0 0x7fb374159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.079+0000 7fb378c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3741f2140 con 0x7fb3740b5a30 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.079+0000 7fb378c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3741f6680 con 0x7fb37415cda0 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.079+0000 7fb378c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3741f23c0 con 0x7fb3740b4350 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb369400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3740b5a30 0x7fb3740b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b4350 0x7fb3741f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b4350 0x7fb3741f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:51396/0 (socket says 172.21.15.105:51396) 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 -- 172.21.15.105:0/457487693 learned_addr learned my addr 172.21.15.105:0/457487693 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb372800640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb37415cda0 0x7fb374159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 -- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb37415cda0 msgr2=0x7fb374159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb37415cda0 0x7fb374159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 -- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3740b5a30 msgr2=0x7fb3740b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3740b5a30 0x7fb3740b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.080+0000 7fb371e00640 1 -- 172.21.15.105:0/457487693 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c009a70 con 0x7fb3740b4350 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.081+0000 7fb371e00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b4350 0x7fb3741f6140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb36400c410 tx=0x7fb36400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.081+0000 7fb36aa00640 1 -- 172.21.15.105:0/457487693 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364005ce0 con 0x7fb3740b4350 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.081+0000 7fb36aa00640 1 -- 172.21.15.105:0/457487693 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb364005e40 con 0x7fb3740b4350 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.081+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3741f26a0 con 0x7fb3740b4350 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.081+0000 7fb36aa00640 1 -- 172.21.15.105:0/457487693 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3640072f0 con 0x7fb3740b4350 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.081+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3741f2c60 con 0x7fb3740b4350 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.082+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb374073390 con 0x7fb3740b4350 2024-03-23T07:56:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.083+0000 7fb36aa00640 1 -- 172.21.15.105:0/457487693 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb364007450 con 0x7fb3740b4350 2024-03-23T07:56:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.084+0000 7fb36aa00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb35006ac30 0x7fb35006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.084+0000 7fb36aa00640 1 -- 172.21.15.105:0/457487693 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb364088660 con 0x7fb3740b4350 2024-03-23T07:56:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.084+0000 7fb369400640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb35006ac30 0x7fb35006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.085+0000 7fb369400640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb35006ac30 0x7fb35006d0f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3740b2f80 tx=0x7fb35c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.089+0000 7fb36aa00640 1 -- 172.21.15.105:0/457487693 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb364053c60 con 0x7fb3740b4350 2024-03-23T07:56:13.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.283+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3740b3ed0 con 0x7fb3740b4350 2024-03-23T07:56:13.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.285+0000 7fb36aa00640 1 -- 172.21.15.105:0/457487693 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12430 (secure 0 0 0) 0x7fb364057280 con 0x7fb3740b4350 2024-03-23T07:56:13.285 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:13.285 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:08.391996+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb35006ac30 msgr2=0x7fb35006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb35006ac30 0x7fb35006d0f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3740b2f80 tx=0x7fb35c009190 comp rx=0 tx=0).stop 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b4350 msgr2=0x7fb3741f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b4350 0x7fb3741f6140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb36400c410 tx=0x7fb36400c8d0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 shutdown_connections 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb37415cda0 0x7fb374159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3740b5a30 0x7fb3740b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb35006ac30 0x7fb35006d0f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 --2- 172.21.15.105:0/457487693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3740b4350 0x7fb3741f6140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 >> 172.21.15.105:0/457487693 conn(0x7fb3740cf350 msgr2=0x7fb3740cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 shutdown_connections 2024-03-23T07:56:13.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.288+0000 7fb378c00640 1 -- 172.21.15.105:0/457487693 wait complete. 2024-03-23T07:56:13.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T07:56:13.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:56:13.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.514+0000 7f0e15800640 1 Processor -- start 2024-03-23T07:56:13.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.514+0000 7f0e15800640 1 -- start start 2024-03-23T07:56:13.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.516+0000 7f0e15800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0e100cf7c0 0x7f0e100cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.516+0000 7f0e15800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e100ccd90 con 0x7f0e10160e90 2024-03-23T07:56:13.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.516+0000 7f0e15800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e100b7fc0 con 0x7f0e1015c820 2024-03-23T07:56:13.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.516+0000 7f0e15800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e100cdd70 con 0x7f0e100cf7c0 2024-03-23T07:56:13.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.516+0000 7f0e14e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0e10160e90 0x7f0e100d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39308/0 (socket says 172.21.15.105:39308) 2024-03-23T07:56:13.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.516+0000 7f0e14e00640 1 -- 172.21.15.105:0/2081191816 learned_addr learned my addr 172.21.15.105:0/2081191816 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e07400640 1 --2- 172.21.15.105:0/2081191816 >> v2:172.21.15.132:3300/0 conn(0x7f0e100cf7c0 0x7f0e100cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545853148 0 0) 0x7f0e100ccd90 con 0x7f0e10160e90 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df4003520 con 0x7f0e10160e90 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 333861308 0 0) 0x7f0df4003520 con 0x7f0e10160e90 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 >> v2:172.21.15.132:3300/0 conn(0x7f0e100cf7c0 msgr2=0x7f0e100cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 --2- 172.21.15.105:0/2081191816 >> v2:172.21.15.132:3300/0 conn(0x7f0e100cf7c0 0x7f0e100cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 >> v1:172.21.15.132:6789/0 conn(0x7f0e1015c820 legacy=0x7f0e100cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100b82a0 con 0x7f0e10160e90 2024-03-23T07:56:13.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f0e08001670 con 0x7f0e10160e90 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 >> v1:172.21.15.105:6789/0 conn(0x7f0e10160e90 legacy=0x7f0e100d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0df4003970 0x7f0df4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df40047c0 0x7f0df4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0df400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100cdd70 con 0x7f0df4005180 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400c0b0 con 0x7f0df40047c0 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400c270 con 0x7f0df4003970 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.517+0000 7f0e07400640 1 --2- 172.21.15.105:0/2081191816 >> v2:172.21.15.132:3300/0 conn(0x7f0e100cf7c0 0x7f0e100cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.518+0000 7f0e14e00640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0df400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.518+0000 7f0e0fe00640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0df4003970 0x7f0df4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.518+0000 7f0e14e00640 1 -- 172.21.15.105:0/2081191816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0df4003970 msgr2=0x7f0df4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.518+0000 7f0e14e00640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0df4003970 0x7f0df4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.518+0000 7f0e14e00640 1 -- 172.21.15.105:0/2081191816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df40047c0 msgr2=0x7f0df4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:13.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.518+0000 7f0e14e00640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0df40047c0 0x7f0df4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.518+0000 7f0e14e00640 1 -- 172.21.15.105:0/2081191816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4003520 con 0x7f0df4005180 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.519+0000 7f0e14e00640 1 -- 172.21.15.105:0/2081191816 reap_dead start 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.519+0000 7f0e14e00640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0df400b980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c240 tx=0x7f0df40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911cab2d1de3e266 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.519+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08005ad0 con 0x7f0df4005180 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.519+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e08005c70 con 0x7f0df4005180 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.519+0000 7f0e0f400640 1 -- 172.21.15.105:0/2081191816 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0802c8b0 con 0x7f0df4005180 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.520+0000 7f0e15800640 1 -- 172.21.15.105:0/2081191816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 msgr2=0x7f0df400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.520+0000 7f0e15800640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0df400b980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c240 tx=0x7f0df40057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.520+0000 7f0e15800640 1 -- 172.21.15.105:0/2081191816 shutdown_connections 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.520+0000 7f0e15800640 1 --2- 172.21.15.105:0/2081191816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0df400b980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.520+0000 7f0e15800640 1 -- 172.21.15.105:0/2081191816 >> 172.21.15.105:0/2081191816 conn(0x7f0e1006b050 msgr2=0x7f0e1006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:13.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.520+0000 7f0e15800640 1 -- 172.21.15.105:0/2081191816 shutdown_connections 2024-03-23T07:56:13.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.520+0000 7f0e15800640 1 -- 172.21.15.105:0/2081191816 wait complete. 2024-03-23T07:56:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.522+0000 7f0e15800640 1 Processor -- start 2024-03-23T07:56:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.522+0000 7f0e15800640 1 -- start start 2024-03-23T07:56:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e15800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0e101f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e15800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e100b4420 0x7f0e100b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e15800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0e100b2d90 0x7f0e100c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e15800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100b23c0 con 0x7f0df4005180 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e15800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e101f6290 con 0x7f0e100b4420 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e15800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e101f6570 con 0x7f0e100b2d90 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0e101f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0e101f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56902/0 (socket says 172.21.15.105:56902) 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.523+0000 7f0e0fe00640 1 -- 172.21.15.105:0/2386532031 learned_addr learned my addr 172.21.15.105:0/2386532031 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e14e00640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0e100b2d90 0x7f0e100c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e07400640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e100b4420 0x7f0e100b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0fe00640 1 -- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0e100b2d90 msgr2=0x7f0e100c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0fe00640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0e100b2d90 0x7f0e100c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0fe00640 1 -- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e100b4420 msgr2=0x7f0e100b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0fe00640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e100b4420 0x7f0e100b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0fe00640 1 -- 172.21.15.105:0/2386532031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e101f6850 con 0x7f0df4005180 2024-03-23T07:56:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e07400640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e100b4420 0x7f0e100b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0fe00640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0e101f5990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d860 tx=0x7f0e0000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0cc00640 1 -- 172.21.15.105:0/2386532031 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00003da0 con 0x7f0df4005180 2024-03-23T07:56:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0cc00640 1 -- 172.21.15.105:0/2386532031 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000b6b0 con 0x7f0df4005180 2024-03-23T07:56:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e0cc00640 1 -- 172.21.15.105:0/2386532031 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000baa0 con 0x7f0df4005180 2024-03-23T07:56:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e100b2050 con 0x7f0df4005180 2024-03-23T07:56:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.524+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e100cc310 con 0x7f0df4005180 2024-03-23T07:56:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.525+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1004e030 con 0x7f0df4005180 2024-03-23T07:56:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.526+0000 7f0e0cc00640 1 -- 172.21.15.105:0/2386532031 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0e0000bc00 con 0x7f0df4005180 2024-03-23T07:56:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.528+0000 7f0e0cc00640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0dec06ac00 0x7f0dec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.528+0000 7f0e0cc00640 1 -- 172.21.15.105:0/2386532031 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0e00089230 con 0x7f0df4005180 2024-03-23T07:56:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.528+0000 7f0e07400640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0dec06ac00 0x7f0dec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.529+0000 7f0e07400640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0dec06ac00 0x7f0dec06d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0df8006fd0 tx=0x7f0df8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.533+0000 7f0e0cc00640 1 -- 172.21.15.105:0/2386532031 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0e00054760 con 0x7f0df4005180 2024-03-23T07:56:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.727+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e100cbfd0 con 0x7f0df4005180 2024-03-23T07:56:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.728+0000 7f0e0cc00640 1 -- 172.21.15.105:0/2386532031 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e0005e070 con 0x7f0df4005180 2024-03-23T07:56:13.727 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:56:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0dec06ac00 msgr2=0x7f0dec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0dec06ac00 0x7f0dec06d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0df8006fd0 tx=0x7f0df8008040 comp rx=0 tx=0).stop 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 msgr2=0x7f0e101f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0e101f5990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d860 tx=0x7f0e0000dd20 comp rx=0 tx=0).stop 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 shutdown_connections 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0dec06ac00 0x7f0dec06d0c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0e100b2d90 0x7f0e100c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e100b4420 0x7f0e100b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 --2- 172.21.15.105:0/2386532031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df4005180 0x7f0e101f5990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.730+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 >> 172.21.15.105:0/2386532031 conn(0x7f0e1006b050 msgr2=0x7f0e100b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.731+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 shutdown_connections 2024-03-23T07:56:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.731+0000 7f0e15800640 1 -- 172.21.15.105:0/2386532031 wait complete. 2024-03-23T07:56:13.746 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:56:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.952+0000 7f879d000640 1 Processor -- start 2024-03-23T07:56:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.952+0000 7f879d000640 1 -- start start 2024-03-23T07:56:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.954+0000 7f879d000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f879815cda0 0x7f87980cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.954+0000 7f879d000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87980d1830 con 0x7f87980b4a60 2024-03-23T07:56:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.954+0000 7f879d000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87980b1f20 con 0x7f87980b5a30 2024-03-23T07:56:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.954+0000 7f879d000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87980d0c20 con 0x7f879815cda0 2024-03-23T07:56:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.954+0000 7f8796800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f87980b4a60 0x7f87980d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39316/0 (socket says 172.21.15.105:39316) 2024-03-23T07:56:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.954+0000 7f8796800640 1 -- 172.21.15.105:0/3378427420 learned_addr learned my addr 172.21.15.105:0/3378427420 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8795e00640 1 --2- 172.21.15.105:0/3378427420 >> v2:172.21.15.132:3300/0 conn(0x7f879815cda0 0x7f87980cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739246486 0 0) 0x7f87980d1830 con 0x7f87980b4a60 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f877c003520 con 0x7f87980b4a60 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878219358 0 0) 0x7f877c003520 con 0x7f87980b4a60 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 >> v2:172.21.15.132:3300/0 conn(0x7f879815cda0 msgr2=0x7f87980cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 --2- 172.21.15.105:0/3378427420 >> v2:172.21.15.132:3300/0 conn(0x7f879815cda0 0x7f87980cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 >> v1:172.21.15.132:6789/0 conn(0x7f87980b5a30 legacy=0x7f87980b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980d0f00 con 0x7f87980b4a60 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f87900019f0 con 0x7f87980b4a60 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8795e00640 1 --2- 172.21.15.105:0/3378427420 >> v2:172.21.15.132:3300/0 conn(0x7f879815cda0 0x7f87980cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 >> v1:172.21.15.105:6789/0 conn(0x7f87980b4a60 legacy=0x7f87980d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f877c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c0047c0 0x7f877c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877c005180 0x7f877c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87980d0c20 con 0x7f877c003970 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c00c0d0 con 0x7f877c0047c0 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.955+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c00c290 con 0x7f877c005180 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8795e00640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f877c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8795400640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c0047c0 0x7f877c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8796800640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877c005180 0x7f877c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8795e00640 1 -- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877c005180 msgr2=0x7f877c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8795e00640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877c005180 0x7f877c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8795e00640 1 -- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c0047c0 msgr2=0x7f877c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8795e00640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877c0047c0 0x7f877c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8795e00640 1 -- 172.21.15.105:0/3378427420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c003520 con 0x7f877c003970 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.956+0000 7f8796800640 1 -- 172.21.15.105:0/3378427420 reap_dead start 2024-03-23T07:56:13.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.957+0000 7f8795e00640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f877c003d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87880098a0 tx=0x7f877c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58799ed4cf7049ec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.957+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788004230 con 0x7f877c003970 2024-03-23T07:56:13.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.957+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788004390 con 0x7f877c003970 2024-03-23T07:56:13.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.957+0000 7f8794a00640 1 -- 172.21.15.105:0/3378427420 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788003830 con 0x7f877c003970 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.958+0000 7f879d000640 1 -- 172.21.15.105:0/3378427420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 msgr2=0x7f877c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.958+0000 7f879d000640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f877c003d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87880098a0 tx=0x7f877c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.958+0000 7f879d000640 1 -- 172.21.15.105:0/3378427420 shutdown_connections 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.958+0000 7f879d000640 1 --2- 172.21.15.105:0/3378427420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f877c003d90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.958+0000 7f879d000640 1 -- 172.21.15.105:0/3378427420 >> 172.21.15.105:0/3378427420 conn(0x7f87980cf350 msgr2=0x7f87980cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.958+0000 7f879d000640 1 -- 172.21.15.105:0/3378427420 shutdown_connections 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.958+0000 7f879d000640 1 -- 172.21.15.105:0/3378427420 wait complete. 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.960+0000 7f879d000640 1 Processor -- start 2024-03-23T07:56:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.960+0000 7f879d000640 1 -- start start 2024-03-23T07:56:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f879d000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f879815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f879d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87980d2e80 0x7f87980cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f879d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87980d3300 0x7f87980cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f879d000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879815dbe0 con 0x7f877c003970 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f879d000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87980cbb40 con 0x7f87980d2e80 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f879d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87980cbdd0 con 0x7f87980d3300 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f879815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f879815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56916/0 (socket says 172.21.15.105:56916) 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 -- 172.21.15.105:0/2007508250 learned_addr learned my addr 172.21.15.105:0/2007508250 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8796800640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87980d3300 0x7f87980cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 -- 172.21.15.105:0/2007508250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87980d3300 msgr2=0x7f87980cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87980d3300 0x7f87980cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 -- 172.21.15.105:0/2007508250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87980d2e80 msgr2=0x7f87980cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87980d2e80 0x7f87980cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.961+0000 7f8795e00640 1 -- 172.21.15.105:0/2007508250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879815d2b0 con 0x7f877c003970 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.962+0000 7f8795e00640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f879815e070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87880098a0 tx=0x7f8788004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.962+0000 7f878e000640 1 -- 172.21.15.105:0/2007508250 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788030610 con 0x7f877c003970 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.962+0000 7f878e000640 1 -- 172.21.15.105:0/2007508250 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788030770 con 0x7f877c003970 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.962+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879815d590 con 0x7f877c003970 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.962+0000 7f878e000640 1 -- 172.21.15.105:0/2007508250 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788030ae0 con 0x7f877c003970 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.962+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87980c99a0 con 0x7f877c003970 2024-03-23T07:56:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.963+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8798073390 con 0x7f877c003970 2024-03-23T07:56:13.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.964+0000 7f878e000640 1 -- 172.21.15.105:0/2007508250 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8788030c40 con 0x7f877c003970 2024-03-23T07:56:13.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.965+0000 7f878e000640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f877806ac30 0x7f877806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:13.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.965+0000 7f878e000640 1 -- 172.21.15.105:0/2007508250 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87880aaab0 con 0x7f877c003970 2024-03-23T07:56:13.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.965+0000 7f8795400640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f877806ac30 0x7f877806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:13.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.966+0000 7f8795400640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f877806ac30 0x7f877806d0f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8784000c00 tx=0x7f8784009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:13.969+0000 7f878e000640 1 -- 172.21.15.105:0/2007508250 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8788076100 con 0x7f877c003970 2024-03-23T07:56:14.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.202+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f879815dbe0 con 0x7f877c003970 2024-03-23T07:56:14.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.203+0000 7f878e000640 1 -- 172.21.15.105:0/2007508250 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3533 (secure 0 0 0) 0x7f8788048020 con 0x7f877c003970 2024-03-23T07:56:14.203 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:14.203 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-23T07:56:14.203 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2024-03-23T07:56:14.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f877806ac30 msgr2=0x7f877806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f877806ac30 0x7f877806d0f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8784000c00 tx=0x7f8784009040 comp rx=0 tx=0).stop 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 msgr2=0x7f879815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f879815e070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87880098a0 tx=0x7f8788004730 comp rx=0 tx=0).stop 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 shutdown_connections 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f877806ac30 0x7f877806d0f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87980d3300 0x7f87980cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87980d2e80 0x7f87980cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 --2- 172.21.15.105:0/2007508250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f877c003970 0x7f879815e070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 >> 172.21.15.105:0/2007508250 conn(0x7f87980cf350 msgr2=0x7f87980b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 shutdown_connections 2024-03-23T07:56:14.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.206+0000 7f879d000640 1 -- 172.21.15.105:0/2007508250 wait complete. 2024-03-23T07:56:14.223 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:14.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.416+0000 7f902e800640 1 Processor -- start 2024-03-23T07:56:14.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.416+0000 7f902e800640 1 -- start start 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.418+0000 7f902e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 0x7f9028167880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.418+0000 7f902e800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9028161640 con 0x7f902813faa0 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.418+0000 7f902e800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90280b38a0 con 0x7f9028160800 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.418+0000 7f902e800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90280b3b80 con 0x7f90280b1db0 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.418+0000 7f9027400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f902813faa0 0x7f9028160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39326/0 (socket says 172.21.15.105:39326) 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.418+0000 7f9027400640 1 -- 172.21.15.105:0/889761955 learned_addr learned my addr 172.21.15.105:0/889761955 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9027e00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 0x7f9028167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385622054 0 0) 0x7f9028161640 con 0x7f902813faa0 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900c003520 con 0x7f902813faa0 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1592831805 0 0) 0x7f900c003520 con 0x7f902813faa0 2024-03-23T07:56:14.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 msgr2=0x7f9028167880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 0x7f9028167880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 >> v1:172.21.15.132:6789/0 conn(0x7f9028160800 legacy=0x7f90280b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90280b3e60 con 0x7f902813faa0 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f9018001670 con 0x7f902813faa0 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 >> v1:172.21.15.105:6789/0 conn(0x7f902813faa0 legacy=0x7f9028160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f900c0039a0 0x7f900c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 0x7f900c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f900c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90280b3b80 con 0x7f900c004d00 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00bb90 con 0x7f900c0039a0 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00bd50 con 0x7f90280b1db0 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.419+0000 7f9027e00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f900c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f9027400640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f900c0039a0 0x7f900c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f9027e00640 1 -- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 msgr2=0x7f900c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f901ea00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 0x7f900c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f9027e00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 0x7f900c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f9027e00640 1 -- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f900c0039a0 msgr2=0x7f900c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f9027e00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f900c0039a0 0x7f900c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f9027e00640 1 -- 172.21.15.105:0/889761955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c003520 con 0x7f900c004d00 2024-03-23T07:56:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f9027400640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f900c0039a0 0x7f900c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.420+0000 7f901ea00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90280b1db0 0x7f900c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.421+0000 7f9027e00640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f900c005180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9020007330 tx=0x7f900c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7111490f8763c934 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.421+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902002cca0 con 0x7f900c004d00 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.421+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902002ce00 con 0x7f900c004d00 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.421+0000 7f9026a00640 1 -- 172.21.15.105:0/889761955 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020034680 con 0x7f900c004d00 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.421+0000 7f902e800640 1 -- 172.21.15.105:0/889761955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 msgr2=0x7f900c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.421+0000 7f902e800640 1 --2- 172.21.15.105:0/889761955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f900c005180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9020007330 tx=0x7f900c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.421+0000 7f9027e00640 1 -- 172.21.15.105:0/889761955 reap_dead start 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.422+0000 7f902e800640 1 -- 172.21.15.105:0/889761955 shutdown_connections 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.422+0000 7f902e800640 1 -- 172.21.15.105:0/889761955 >> 172.21.15.105:0/889761955 conn(0x7f902806a3d0 msgr2=0x7f90280b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:14.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.422+0000 7f902e800640 1 -- 172.21.15.105:0/889761955 shutdown_connections 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.422+0000 7f902e800640 1 -- 172.21.15.105:0/889761955 wait complete. 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.423+0000 7f902e800640 1 Processor -- start 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.423+0000 7f902e800640 1 -- start start 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.424+0000 7f902e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f902815f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.424+0000 7f902e800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90280cc590 0x7f9028153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.424+0000 7f902e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f902813ead0 0x7f902813ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.424+0000 7f902e800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902813f4c0 con 0x7f900c004d00 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.424+0000 7f902e800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028153460 con 0x7f90280cc590 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.424+0000 7f902e800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90280cd630 con 0x7f902813ead0 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f902815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f902815f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56938/0 (socket says 172.21.15.105:56938) 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 -- 172.21.15.105:0/228026569 learned_addr learned my addr 172.21.15.105:0/228026569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 -- 172.21.15.105:0/228026569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f902813ead0 msgr2=0x7f902813ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f902813ead0 0x7f902813ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 -- 172.21.15.105:0/228026569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90280cc590 msgr2=0x7f9028153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90280cc590 0x7f9028153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 -- 172.21.15.105:0/228026569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90280cca20 con 0x7f900c004d00 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f9027400640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f902815f140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f901800d7c0 tx=0x7f901800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f901fe00640 1 -- 172.21.15.105:0/228026569 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90180074e0 con 0x7f900c004d00 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f901fe00640 1 -- 172.21.15.105:0/228026569 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9018007640 con 0x7f900c004d00 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.425+0000 7f901fe00640 1 -- 172.21.15.105:0/228026569 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90180049e0 con 0x7f900c004d00 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.426+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90280ccd00 con 0x7f900c004d00 2024-03-23T07:56:14.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.426+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90280cd2c0 con 0x7f900c004d00 2024-03-23T07:56:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.427+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9028073390 con 0x7f900c004d00 2024-03-23T07:56:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.428+0000 7f901fe00640 1 -- 172.21.15.105:0/228026569 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9018004b40 con 0x7f900c004d00 2024-03-23T07:56:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.428+0000 7f901fe00640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f900406ac00 0x7f900406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.428+0000 7f901fe00640 1 -- 172.21.15.105:0/228026569 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9018088b50 con 0x7f900c004d00 2024-03-23T07:56:14.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.429+0000 7f901ea00640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f900406ac00 0x7f900406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.429+0000 7f901ea00640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f900406ac00 0x7f900406d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9010000f60 tx=0x7f9010009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:14.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.433+0000 7f901fe00640 1 -- 172.21.15.105:0/228026569 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9018053f50 con 0x7f900c004d00 2024-03-23T07:56:14.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.613+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f902815c660 con 0x7f900c004d00 2024-03-23T07:56:14.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.614+0000 7f901fe00640 1 -- 172.21.15.105:0/228026569 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f9018019070 con 0x7f900c004d00 2024-03-23T07:56:14.614 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:14.614 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:08.391996+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:14.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f900406ac00 msgr2=0x7f900406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f900406ac00 0x7f900406d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9010000f60 tx=0x7f9010009040 comp rx=0 tx=0).stop 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 msgr2=0x7f902815f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f902815f140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f901800d7c0 tx=0x7f901800dc80 comp rx=0 tx=0).stop 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 shutdown_connections 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f900406ac00 0x7f900406d0c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f902813ead0 0x7f902813ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90280cc590 0x7f9028153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 --2- 172.21.15.105:0/228026569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900c004d00 0x7f902815f140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 >> 172.21.15.105:0/228026569 conn(0x7f902806a3d0 msgr2=0x7f90280afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 shutdown_connections 2024-03-23T07:56:14.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.617+0000 7f902e800640 1 -- 172.21.15.105:0/228026569 wait complete. 2024-03-23T07:56:14.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T07:56:14.633 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:56:14.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.840+0000 7f1300200640 1 Processor -- start 2024-03-23T07:56:14.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.840+0000 7f1300200640 1 -- start start 2024-03-23T07:56:14.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.841+0000 7f1300200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12f8159830 0x7f12f80cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.841+0000 7f1300200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 0x7f12f80cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f1300200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f8159c10 con 0x7f12f80ca7b0 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f1300200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f806de50 con 0x7f12f8159830 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f1300200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f80cf300 con 0x7f12f80ce930 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12fce00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f12f80ca7b0 0x7f12f80cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39336/0 (socket says 172.21.15.105:39336) 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12fce00640 1 -- 172.21.15.105:0/2841123432 learned_addr learned my addr 172.21.15.105:0/2841123432 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12fd800640 1 --2- 172.21.15.105:0/2841123432 >> v2:172.21.15.132:3300/0 conn(0x7f12f8159830 0x7f12f80cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90066092 0 0) 0x7f12f8159c10 con 0x7f12f80ca7b0 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e0003520 con 0x7f12f80ca7b0 2024-03-23T07:56:14.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12fe200640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 0x7f12f80cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538762488 0 0) 0x7f12e0003520 con 0x7f12f80ca7b0 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 msgr2=0x7f12f80cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12f7e00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 0x7f12f80cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 >> v2:172.21.15.132:3300/0 conn(0x7f12f8159830 msgr2=0x7f12f80cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.842+0000 7f12f7e00640 1 --2- 172.21.15.105:0/2841123432 >> v2:172.21.15.132:3300/0 conn(0x7f12f8159830 0x7f12f80cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80cd840 con 0x7f12f80ca7b0 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f12e8001670 con 0x7f12f80ca7b0 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fe200640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 0x7f12f80cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:14.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 >> v1:172.21.15.105:6789/0 conn(0x7f12f80ca7b0 legacy=0x7f12f80cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12e0003970 0x7f12e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e00047c0 0x7f12e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 0x7f12e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f80cf300 con 0x7f12e00047c0 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e000b9e0 con 0x7f12e0003970 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e000ddb0 con 0x7f12f80ce930 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fce00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e00047c0 0x7f12e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fd800640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12e0003970 0x7f12e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fe200640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 0x7f12e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fce00640 1 -- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 msgr2=0x7f12e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fce00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12f80ce930 0x7f12e0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fce00640 1 -- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12e0003970 msgr2=0x7f12e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fce00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12e0003970 0x7f12e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fce00640 1 -- 172.21.15.105:0/2841123432 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e0003520 con 0x7f12e00047c0 2024-03-23T07:56:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.843+0000 7f12fd800640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12e0003970 0x7f12e0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.844+0000 7f12fce00640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e00047c0 0x7f12e0004c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c110 tx=0x7f12e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4326b3b4375ec43 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8005ad0 con 0x7f12e00047c0 2024-03-23T07:56:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e8005c50 con 0x7f12e00047c0 2024-03-23T07:56:14.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f12f7e00640 1 -- 172.21.15.105:0/2841123432 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e802c8b0 con 0x7f12e00047c0 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f1300200640 1 -- 172.21.15.105:0/2841123432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e00047c0 msgr2=0x7f12e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f1300200640 1 --2- 172.21.15.105:0/2841123432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e00047c0 0x7f12e0004c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c110 tx=0x7f12e00057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f12fe200640 1 -- 172.21.15.105:0/2841123432 reap_dead start 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f1300200640 1 -- 172.21.15.105:0/2841123432 shutdown_connections 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f1300200640 1 -- 172.21.15.105:0/2841123432 >> 172.21.15.105:0/2841123432 conn(0x7f12f806a0b0 msgr2=0x7f12f806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f1300200640 1 -- 172.21.15.105:0/2841123432 shutdown_connections 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.845+0000 7f1300200640 1 -- 172.21.15.105:0/2841123432 wait complete. 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.847+0000 7f1300200640 1 Processor -- start 2024-03-23T07:56:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.847+0000 7f1300200640 1 -- start start 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f1300200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12e00047c0 0x7f12f814e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f1300200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f814eb20 0x7f12f8152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f1300200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12f8151090 0x7f12f815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f1300200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f80c7dc0 con 0x7f12f814eb20 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f1300200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f8153cd0 con 0x7f12f8151090 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f1300200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f8153490 con 0x7f12e00047c0 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f12fce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f814eb20 0x7f12f8152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f12fce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f814eb20 0x7f12f8152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56954/0 (socket says 172.21.15.105:56954) 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f12fce00640 1 -- 172.21.15.105:0/2025036980 learned_addr learned my addr 172.21.15.105:0/2025036980 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f12fd800640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12e00047c0 0x7f12f814e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f12fe200640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12f8151090 0x7f12f815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f12fce00640 1 -- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12e00047c0 msgr2=0x7f12f814e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.848+0000 7f12fce00640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12e00047c0 0x7f12f814e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12fce00640 1 -- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12f8151090 msgr2=0x7f12f815b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12fce00640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12f8151090 0x7f12f815b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12fce00640 1 -- 172.21.15.105:0/2025036980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f8153710 con 0x7f12f814eb20 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12fd800640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12e00047c0 0x7f12f814e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12fce00640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f814eb20 0x7f12f8152f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12e802c520 tx=0x7f12e8001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12f5600640 1 -- 172.21.15.105:0/2025036980 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e803c040 con 0x7f12f814eb20 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12f5600640 1 -- 172.21.15.105:0/2025036980 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e802dd50 con 0x7f12f814eb20 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f12f5600640 1 -- 172.21.15.105:0/2025036980 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e80072b0 con 0x7f12f814eb20 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f81539f0 con 0x7f12f814eb20 2024-03-23T07:56:14.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.849+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f815b8f0 con 0x7f12f814eb20 2024-03-23T07:56:14.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.850+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f80c7f70 con 0x7f12f814eb20 2024-03-23T07:56:14.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.851+0000 7f12f5600640 1 -- 172.21.15.105:0/2025036980 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f12e8032070 con 0x7f12f814eb20 2024-03-23T07:56:14.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.852+0000 7f12f5600640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f12dc06ac00 0x7f12dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:14.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.852+0000 7f12f5600640 1 -- 172.21.15.105:0/2025036980 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12e80aab30 con 0x7f12f814eb20 2024-03-23T07:56:14.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.852+0000 7f12fd800640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f12dc06ac00 0x7f12dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:14.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.853+0000 7f12fd800640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f12dc06ac00 0x7f12dc06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009b00 tx=0x7f12ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:14.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:14.856+0000 7f12f5600640 1 -- 172.21.15.105:0/2025036980 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f12f80c7f70 con 0x7f12f814eb20 2024-03-23T07:56:15.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.034+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12f80c7f70 con 0x7f12f814eb20 2024-03-23T07:56:15.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.035+0000 7f12f5600640 1 -- 172.21.15.105:0/2025036980 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12f80c7f70 con 0x7f12f814eb20 2024-03-23T07:56:15.035 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f12dc06ac00 msgr2=0x7f12dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f12dc06ac00 0x7f12dc06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009b00 tx=0x7f12ec008040 comp rx=0 tx=0).stop 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f814eb20 msgr2=0x7f12f8152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f814eb20 0x7f12f8152f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12e802c520 tx=0x7f12e8001930 comp rx=0 tx=0).stop 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 shutdown_connections 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12f8151090 0x7f12f815b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f814eb20 0x7f12f8152f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f12dc06ac00 0x7f12dc06d0c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 --2- 172.21.15.105:0/2025036980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12e00047c0 0x7f12f814e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 >> 172.21.15.105:0/2025036980 conn(0x7f12f806a0b0 msgr2=0x7f12f80b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 shutdown_connections 2024-03-23T07:56:15.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.038+0000 7f1300200640 1 -- 172.21.15.105:0/2025036980 wait complete. 2024-03-23T07:56:15.054 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T07:56:15.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.258+0000 7f6d35800640 1 Processor -- start 2024-03-23T07:56:15.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.259+0000 7f6d35800640 1 -- start start 2024-03-23T07:56:15.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.260+0000 7f6d35800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6d300cbab0 0x7f6d300b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.260+0000 7f6d35800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3006d390 0x7f6d3006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.260+0000 7f6d35800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d300b16a0 con 0x7f6d300adbc0 2024-03-23T07:56:15.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.260+0000 7f6d35800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3006dd60 con 0x7f6d300cbab0 2024-03-23T07:56:15.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.260+0000 7f6d35800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d300cbfb0 con 0x7f6d3006d390 2024-03-23T07:56:15.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.260+0000 7f6d2ea00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6d300adbc0 0x7f6d300b41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39342/0 (socket says 172.21.15.105:39342) 2024-03-23T07:56:15.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.260+0000 7f6d2ea00640 1 -- 172.21.15.105:0/3053789910 learned_addr learned my addr 172.21.15.105:0/3053789910 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d26000640 1 --2- 172.21.15.105:0/3053789910 >> v2:172.21.15.132:3300/0 conn(0x7f6d300cbab0 0x7f6d300b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328862293 0 0) 0x7f6d300b16a0 con 0x7f6d300adbc0 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2f400640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3006d390 0x7f6d3006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d14003520 con 0x7f6d300adbc0 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1391912538 0 0) 0x7f6d14003520 con 0x7f6d300adbc0 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3006d390 msgr2=0x7f6d3006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3006d390 0x7f6d3006d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 >> v2:172.21.15.132:3300/0 conn(0x7f6d300cbab0 msgr2=0x7f6d300b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:15.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 --2- 172.21.15.105:0/3053789910 >> v2:172.21.15.132:3300/0 conn(0x7f6d300cbab0 0x7f6d300b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d300cc290 con 0x7f6d300adbc0 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f6d20001670 con 0x7f6d300adbc0 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 >> v1:172.21.15.105:6789/0 conn(0x7f6d300adbc0 legacy=0x7f6d300b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140047c0 0x7f6d14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.261+0000 7f6d2e000640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3006d390 0x7f6d14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3006dd60 con 0x7f6d14003970 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d300cbfb0 con 0x7f6d140047c0 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1400b980 con 0x7f6d3006d390 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2ea00640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2ea00640 1 -- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3006d390 msgr2=0x7f6d14005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2ea00640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3006d390 0x7f6d14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2ea00640 1 -- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140047c0 msgr2=0x7f6d14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2ea00640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140047c0 0x7f6d14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.262+0000 7f6d2ea00640 1 -- 172.21.15.105:0/3053789910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d14003520 con 0x7f6d14003970 2024-03-23T07:56:15.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.263+0000 7f6d2ea00640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d14003d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000c270 tx=0x7f6d140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eae78bf866a732 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.263+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2002cae0 con 0x7f6d14003970 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.263+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2002cc40 con 0x7f6d14003970 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.263+0000 7f6d2e000640 1 -- 172.21.15.105:0/3053789910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200058d0 con 0x7f6d14003970 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.264+0000 7f6d35800640 1 -- 172.21.15.105:0/3053789910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 msgr2=0x7f6d14003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.264+0000 7f6d35800640 1 --2- 172.21.15.105:0/3053789910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d14003d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000c270 tx=0x7f6d140057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.264+0000 7f6d2f400640 1 -- 172.21.15.105:0/3053789910 reap_dead start 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.264+0000 7f6d35800640 1 -- 172.21.15.105:0/3053789910 shutdown_connections 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.264+0000 7f6d35800640 1 -- 172.21.15.105:0/3053789910 >> 172.21.15.105:0/3053789910 conn(0x7f6d3006a3d0 msgr2=0x7f6d300b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.264+0000 7f6d35800640 1 -- 172.21.15.105:0/3053789910 shutdown_connections 2024-03-23T07:56:15.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.264+0000 7f6d35800640 1 -- 172.21.15.105:0/3053789910 wait complete. 2024-03-23T07:56:15.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.266+0000 7f6d35800640 1 Processor -- start 2024-03-23T07:56:15.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.266+0000 7f6d35800640 1 -- start start 2024-03-23T07:56:15.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d35800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d301ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d35800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d300ac200 0x7f6d300ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d35800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3015a940 0x7f6d3015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d35800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3015b8e0 con 0x7f6d14003970 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d35800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30132e40 con 0x7f6d300ac200 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d35800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30130140 con 0x7f6d3015a940 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d301ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d301ef920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56970/0 (socket says 172.21.15.105:56970) 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 -- 172.21.15.105:0/2702578569 learned_addr learned my addr 172.21.15.105:0/2702578569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d26000640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d300ac200 0x7f6d300ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2f400640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3015a940 0x7f6d3015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 -- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3015a940 msgr2=0x7f6d3015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3015a940 0x7f6d3015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 -- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d300ac200 msgr2=0x7f6d300ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d300ac200 0x7f6d300ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2ea00640 1 -- 172.21.15.105:0/2702578569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d300acbc0 con 0x7f6d14003970 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d2f400640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3015a940 0x7f6d3015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.267+0000 7f6d26000640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d300ac200 0x7f6d300ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.268+0000 7f6d2ea00640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d301ef920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000c270 tx=0x7f6d20001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.268+0000 7f6d27400640 1 -- 172.21.15.105:0/2702578569 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20039070 con 0x7f6d14003970 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.268+0000 7f6d27400640 1 -- 172.21.15.105:0/2702578569 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d20001630 con 0x7f6d14003970 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.268+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3015b600 con 0x7f6d14003970 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.268+0000 7f6d27400640 1 -- 172.21.15.105:0/2702578569 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2003c040 con 0x7f6d14003970 2024-03-23T07:56:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.268+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3012fdd0 con 0x7f6d14003970 2024-03-23T07:56:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.269+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3012f950 con 0x7f6d14003970 2024-03-23T07:56:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.270+0000 7f6d27400640 1 -- 172.21.15.105:0/2702578569 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6d20032070 con 0x7f6d14003970 2024-03-23T07:56:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.271+0000 7f6d27400640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6d0c06ac00 0x7f6d0c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.271+0000 7f6d27400640 1 -- 172.21.15.105:0/2702578569 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6d200aa770 con 0x7f6d14003970 2024-03-23T07:56:15.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.271+0000 7f6d26000640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6d0c06ac00 0x7f6d0c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.272+0000 7f6d26000640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6d0c06ac00 0x7f6d0c06d0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d300b1750 tx=0x7f6d1800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:15.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.275+0000 7f6d27400640 1 -- 172.21.15.105:0/2702578569 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6d3012f950 con 0x7f6d14003970 2024-03-23T07:56:15.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.458+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6d3012f950 con 0x7f6d14003970 2024-03-23T07:56:15.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.459+0000 7f6d27400640 1 -- 172.21.15.105:0/2702578569 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6d3012f950 con 0x7f6d14003970 2024-03-23T07:56:15.458 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:15.458 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-03-23T07:56:15.458 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:56:15.458 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-03-23T07:56:15.459 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-03-23T07:56:15.459 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:56:15.459 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6d0c06ac00 msgr2=0x7f6d0c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6d0c06ac00 0x7f6d0c06d0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d300b1750 tx=0x7f6d1800b040 comp rx=0 tx=0).stop 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 msgr2=0x7f6d301ef920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d301ef920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000c270 tx=0x7f6d20001a00 comp rx=0 tx=0).stop 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 shutdown_connections 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6d0c06ac00 0x7f6d0c06d0c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d3015a940 0x7f6d3015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d300ac200 0x7f6d300ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 --2- 172.21.15.105:0/2702578569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d14003970 0x7f6d301ef920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 >> 172.21.15.105:0/2702578569 conn(0x7f6d3006a3d0 msgr2=0x7f6d300cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:15.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 shutdown_connections 2024-03-23T07:56:15.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.461+0000 7f6d35800640 1 -- 172.21.15.105:0/2702578569 wait complete. 2024-03-23T07:56:15.478 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T07:56:15.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T07:56:15.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.692+0000 7fb903600640 1 Processor -- start 2024-03-23T07:56:15.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.692+0000 7fb903600640 1 -- start start 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.693+0000 7fb903600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159830 0x7fb8fc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.693+0000 7fb903600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc0ca7b0 0x7fb8fc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.693+0000 7fb903600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb8fc0ce930 0x7fb8fc0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.693+0000 7fb903600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc06de50 con 0x7fb8fc159830 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.693+0000 7fb903600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc0cf300 con 0x7fb8fc0ce930 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.693+0000 7fb903600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc159c10 con 0x7fb8fc0ca7b0 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159830 0x7fb8fc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159830 0x7fb8fc0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56972/0 (socket says 172.21.15.105:56972) 2024-03-23T07:56:15.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 -- 172.21.15.105:0/4182891630 learned_addr learned my addr 172.21.15.105:0/4182891630 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 -- 172.21.15.105:0/4182891630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc0ca7b0 msgr2=0x7fb8fc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 --2- 172.21.15.105:0/4182891630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc0ca7b0 0x7fb8fc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 -- 172.21.15.105:0/4182891630 >> v2:172.21.15.132:3300/0 conn(0x7fb8fc0ce930 msgr2=0x7fb8fc0cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb901600640 1 --2- 172.21.15.105:0/4182891630 >> v2:172.21.15.132:3300/0 conn(0x7fb8fc0ce930 0x7fb8fc0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 --2- 172.21.15.105:0/4182891630 >> v2:172.21.15.132:3300/0 conn(0x7fb8fc0ce930 0x7fb8fc0cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb900c00640 1 -- 172.21.15.105:0/4182891630 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc0cd840 con 0x7fb8fc159830 2024-03-23T07:56:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.694+0000 7fb901600640 1 --2- 172.21.15.105:0/4182891630 >> v2:172.21.15.132:3300/0 conn(0x7fb8fc0ce930 0x7fb8fc0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.695+0000 7fb900c00640 1 --2- 172.21.15.105:0/4182891630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc159830 0x7fb8fc0cc3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0cb030 tx=0x7fb8ec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f13b397109e762 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.695+0000 7fb8fbe00640 1 -- 172.21.15.105:0/4182891630 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec030860 con 0x7fb8fc159830 2024-03-23T07:56:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb8fbe00640 1 -- 172.21.15.105:0/4182891630 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec0309c0 con 0x7fb8fc159830 2024-03-23T07:56:15.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb8fbe00640 1 -- 172.21.15.105:0/4182891630 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec030d50 con 0x7fb8fc159830 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 -- 172.21.15.105:0/4182891630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc159830 msgr2=0x7fb8fc0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 --2- 172.21.15.105:0/4182891630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc159830 0x7fb8fc0cc3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0cb030 tx=0x7fb8ec02b9a0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 -- 172.21.15.105:0/4182891630 shutdown_connections 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 --2- 172.21.15.105:0/4182891630 >> v2:172.21.15.132:3300/0 conn(0x7fb8fc0ce930 0x7fb8fc0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 --2- 172.21.15.105:0/4182891630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc0ca7b0 0x7fb8fc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 --2- 172.21.15.105:0/4182891630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc159830 0x7fb8fc0cc3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 -- 172.21.15.105:0/4182891630 >> 172.21.15.105:0/4182891630 conn(0x7fb8fc06a0b0 msgr2=0x7fb8fc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:15.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 -- 172.21.15.105:0/4182891630 shutdown_connections 2024-03-23T07:56:15.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.696+0000 7fb903600640 1 -- 172.21.15.105:0/4182891630 wait complete. 2024-03-23T07:56:15.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.698+0000 7fb903600640 1 Processor -- start 2024-03-23T07:56:15.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.698+0000 7fb903600640 1 -- start start 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb903600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8fc0ca7b0 0x7fb8fc150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb903600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc0ce930 0x7fb8fc151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb903600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc159830 0x7fb8fc1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb903600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc154780 con 0x7fb8fc0ce930 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb903600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc151ad0 con 0x7fb8fc0ca7b0 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb903600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc153de0 con 0x7fb8fc159830 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc0ce930 0x7fb8fc151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc0ce930 0x7fb8fc151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56988/0 (socket says 172.21.15.105:56988) 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.699+0000 7fb8f3e00640 1 -- 172.21.15.105:0/3229429852 learned_addr learned my addr 172.21.15.105:0/3229429852 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb900c00640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8fc0ca7b0 0x7fb8fc150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb901600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc159830 0x7fb8fc1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f3e00640 1 -- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc159830 msgr2=0x7fb8fc1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f3e00640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc159830 0x7fb8fc1532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f3e00640 1 -- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8fc0ca7b0 msgr2=0x7fb8fc150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f3e00640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8fc0ca7b0 0x7fb8fc150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f3e00640 1 -- 172.21.15.105:0/3229429852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0094d0 con 0x7fb8fc0ce930 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f3e00640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc0ce930 0x7fb8fc151590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400b4a0 tx=0x7fb8e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f9600640 1 -- 172.21.15.105:0/3229429852 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4010610 con 0x7fb8fc0ce930 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f9600640 1 -- 172.21.15.105:0/3229429852 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e4010770 con 0x7fb8fc0ce930 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.700+0000 7fb8f9600640 1 -- 172.21.15.105:0/3229429852 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4010b20 con 0x7fb8fc0ce930 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.701+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc1540c0 con 0x7fb8fc0ce930 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.701+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc153810 con 0x7fb8fc0ce930 2024-03-23T07:56:15.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.702+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc1548c0 con 0x7fb8fc0ce930 2024-03-23T07:56:15.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.703+0000 7fb8f9600640 1 -- 172.21.15.105:0/3229429852 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb8e4010c80 con 0x7fb8fc0ce930 2024-03-23T07:56:15.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.703+0000 7fb8f9600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb8d806ac00 0x7fb8d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:15.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.703+0000 7fb8f9600640 1 -- 172.21.15.105:0/3229429852 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb8e4088c50 con 0x7fb8fc0ce930 2024-03-23T07:56:15.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.704+0000 7fb900c00640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb8d806ac00 0x7fb8d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:15.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.704+0000 7fb900c00640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb8d806ac00 0x7fb8d806d0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec036000 tx=0x7fb8ec03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:15.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.708+0000 7fb8f9600640 1 -- 172.21.15.105:0/3229429852 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb8fc1548c0 con 0x7fb8fc0ce930 2024-03-23T07:56:15.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:15.901+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb8fc1548c0 con 0x7fb8fc0ce930 2024-03-23T07:56:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.179+0000 7fb8f9600640 1 -- 172.21.15.105:0/3229429852 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb8fc1548c0 con 0x7fb8fc0ce930 2024-03-23T07:56:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-03-23T07:56:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb8d806ac00 msgr2=0x7fb8d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb8d806ac00 0x7fb8d806d0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec036000 tx=0x7fb8ec03b040 comp rx=0 tx=0).stop 2024-03-23T07:56:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc0ce930 msgr2=0x7fb8fc151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc0ce930 0x7fb8fc151590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400b4a0 tx=0x7fb8e400b960 comp rx=0 tx=0).stop 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 shutdown_connections 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb8d806ac00 0x7fb8d806d0c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8fc159830 0x7fb8fc1532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc0ce930 0x7fb8fc151590 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 --2- 172.21.15.105:0/3229429852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8fc0ca7b0 0x7fb8fc150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 >> 172.21.15.105:0/3229429852 conn(0x7fb8fc06a0b0 msgr2=0x7fb8fc0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.184+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 shutdown_connections 2024-03-23T07:56:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.185+0000 7fb903600640 1 -- 172.21.15.105:0/3229429852 wait complete. 2024-03-23T07:56:16.208 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-23T07:56:16.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.402+0000 7f67ce000640 1 Processor -- start 2024-03-23T07:56:16.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.403+0000 7f67ce000640 1 -- start start 2024-03-23T07:56:16.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.404+0000 7f67ce000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f67c80cd740 0x7f67c80b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.404+0000 7f67ce000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c80d20c0 con 0x7f67c80b5aa0 2024-03-23T07:56:16.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.404+0000 7f67ce000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c80d14b0 con 0x7f67c80d0d40 2024-03-23T07:56:16.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.404+0000 7f67ce000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c80d1790 con 0x7f67c80cd740 2024-03-23T07:56:16.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.404+0000 7f67c7e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f67c80b5aa0 0x7f67c80b5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39346/0 (socket says 172.21.15.105:39346) 2024-03-23T07:56:16.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.404+0000 7f67c7e00640 1 -- 172.21.15.105:0/2918929057 learned_addr learned my addr 172.21.15.105:0/2918929057 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:16.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c7400640 1 --2- 172.21.15.105:0/2918929057 >> v2:172.21.15.132:3300/0 conn(0x7f67c80cd740 0x7f67c80b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:16.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809092297 0 0) 0x7f67c80d20c0 con 0x7f67c80b5aa0 2024-03-23T07:56:16.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a8003520 con 0x7f67c80b5aa0 2024-03-23T07:56:16.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1694526563 0 0) 0x7f67a8003520 con 0x7f67c80b5aa0 2024-03-23T07:56:16.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 >> v2:172.21.15.132:3300/0 conn(0x7f67c80cd740 msgr2=0x7f67c80b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:16.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 --2- 172.21.15.105:0/2918929057 >> v2:172.21.15.132:3300/0 conn(0x7f67c80cd740 0x7f67c80b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 >> v1:172.21.15.132:6789/0 conn(0x7f67c80d0d40 legacy=0x7f67c80cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80d1a70 con 0x7f67c80b5aa0 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f67c0001670 con 0x7f67c80b5aa0 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 >> v1:172.21.15.105:6789/0 conn(0x7f67c80b5aa0 legacy=0x7f67c80b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a8003970 0x7f67a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.405+0000 7f67c6000640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6000640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8005180 0x7f67a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80d1790 con 0x7f67a80047c0 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a800c020 con 0x7f67a8005180 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a800c1e0 con 0x7f67a8003970 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6a00640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c7e00640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8005180 0x7f67a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c7400640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a8003970 0x7f67a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6a00640 1 -- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a8003970 msgr2=0x7f67a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6a00640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a8003970 0x7f67a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6a00640 1 -- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8005180 msgr2=0x7f67a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:16.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6a00640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8005180 0x7f67a800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c6a00640 1 -- 172.21.15.105:0/2918929057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8003520 con 0x7f67a80047c0 2024-03-23T07:56:16.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c7e00640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8005180 0x7f67a800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:16.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c7e00640 1 -- 172.21.15.105:0/2918929057 reap_dead start 2024-03-23T07:56:16.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.406+0000 7f67c7400640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a8003970 0x7f67a8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.407+0000 7f67c6a00640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67a8004c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67b0001e50 tx=0x7f67a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a381f2b4c5a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.407+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0003b70 con 0x7f67a80047c0 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.407+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0003cd0 con 0x7f67a80047c0 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.407+0000 7f67c6000640 1 -- 172.21.15.105:0/2918929057 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0005310 con 0x7f67a80047c0 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.408+0000 7f67ce000640 1 -- 172.21.15.105:0/2918929057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 msgr2=0x7f67a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.408+0000 7f67ce000640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67a8004c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67b0001e50 tx=0x7f67a80057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.408+0000 7f67ce000640 1 -- 172.21.15.105:0/2918929057 shutdown_connections 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.408+0000 7f67ce000640 1 --2- 172.21.15.105:0/2918929057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67a8004c40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.408+0000 7f67ce000640 1 -- 172.21.15.105:0/2918929057 >> 172.21.15.105:0/2918929057 conn(0x7f67c80cf5d0 msgr2=0x7f67c80cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:16.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.408+0000 7f67ce000640 1 -- 172.21.15.105:0/2918929057 shutdown_connections 2024-03-23T07:56:16.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.408+0000 7f67ce000640 1 -- 172.21.15.105:0/2918929057 wait complete. 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.410+0000 7f67ce000640 1 Processor -- start 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.410+0000 7f67ce000640 1 -- start start 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67ce000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67c815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67ce000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67c8157970 0x7f67c81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67ce000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67c80c7300 0x7f67c80c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67ce000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c815aef0 con 0x7f67a80047c0 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67ce000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80c6e70 con 0x7f67c8157970 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67ce000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8157720 con 0x7f67c80c7300 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67c7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67c815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67c7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67c815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57006/0 (socket says 172.21.15.105:57006) 2024-03-23T07:56:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.411+0000 7f67c7400640 1 -- 172.21.15.105:0/1950966708 learned_addr learned my addr 172.21.15.105:0/1950966708 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67c7400640 1 -- 172.21.15.105:0/1950966708 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67c80c7300 msgr2=0x7f67c80c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67c7400640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67c80c7300 0x7f67c80c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67c6a00640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67c8157970 0x7f67c81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67c7400640 1 -- 172.21.15.105:0/1950966708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67c8157970 msgr2=0x7f67c81571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67c7400640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67c8157970 0x7f67c81571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67c7400640 1 -- 172.21.15.105:0/1950966708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80c6260 con 0x7f67a80047c0 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67c7400640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67c815a9b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67b80107c0 tx=0x7f67b8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67bf400640 1 -- 172.21.15.105:0/1950966708 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800eb60 con 0x7f67a80047c0 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67bf400640 1 -- 172.21.15.105:0/1950966708 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8010ed0 con 0x7f67a80047c0 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c80c6540 con 0x7f67a80047c0 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.412+0000 7f67bf400640 1 -- 172.21.15.105:0/1950966708 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b801c400 con 0x7f67a80047c0 2024-03-23T07:56:16.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.413+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c80c6b00 con 0x7f67a80047c0 2024-03-23T07:56:16.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.414+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c80c6cc0 con 0x7f67a80047c0 2024-03-23T07:56:16.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.415+0000 7f67bf400640 1 -- 172.21.15.105:0/1950966708 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67b801c560 con 0x7f67a80047c0 2024-03-23T07:56:16.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.416+0000 7f67bf400640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f67a406ac30 0x7f67a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:16.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.416+0000 7f67bf400640 1 -- 172.21.15.105:0/1950966708 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f67b8089610 con 0x7f67a80047c0 2024-03-23T07:56:16.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.416+0000 7f67c6a00640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f67a406ac30 0x7f67a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:16.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.417+0000 7f67c6a00640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f67a406ac30 0x7f67a406d0f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b0034000 tx=0x7f67b0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:16.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.421+0000 7f67bf400640 1 -- 172.21.15.105:0/1950966708 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67b80549c0 con 0x7f67a80047c0 2024-03-23T07:56:16.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:16.614+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f67c8073390 con 0x7f67a80047c0 2024-03-23T07:56:17.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.190+0000 7f67bf400640 1 -- 172.21.15.105:0/1950966708 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f67b8057fe0 con 0x7f67a80047c0 2024-03-23T07:56:17.190 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-03-23T07:56:17.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.196+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f67a406ac30 msgr2=0x7f67a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.196+0000 7f67ce000640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f67a406ac30 0x7f67a406d0f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b0034000 tx=0x7f67b0009030 comp rx=0 tx=0).stop 2024-03-23T07:56:17.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.196+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 msgr2=0x7f67c815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.196+0000 7f67ce000640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67c815a9b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67b80107c0 tx=0x7f67b8010c80 comp rx=0 tx=0).stop 2024-03-23T07:56:17.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.199+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 shutdown_connections 2024-03-23T07:56:17.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.199+0000 7f67ce000640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f67a406ac30 0x7f67a406d0f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.199+0000 7f67ce000640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67c80c7300 0x7f67c80c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.199+0000 7f67ce000640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67c8157970 0x7f67c81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.199+0000 7f67ce000640 1 --2- 172.21.15.105:0/1950966708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67a80047c0 0x7f67c815a9b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.199+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 >> 172.21.15.105:0/1950966708 conn(0x7f67c80cf5d0 msgr2=0x7f67c815bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:17.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.202+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 shutdown_connections 2024-03-23T07:56:17.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.205+0000 7f67ce000640 1 -- 172.21.15.105:0/1950966708 wait complete. 2024-03-23T07:56:17.234 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T07:56:17.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.416+0000 7f4c60800640 1 Processor -- start 2024-03-23T07:56:17.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.416+0000 7f4c60800640 1 -- start start 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c60800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c580cd740 0x7f4c580b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c60800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c580d0d40 0x7f4c580cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c60800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c580d20c0 con 0x7f4c5806b0a0 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c60800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c580ce060 con 0x7f4c580d0d40 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c60800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c580d14b0 con 0x7f4c580cd740 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c5d400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c580d0d40 0x7f4c580cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c5de00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c580cd740 0x7f4c580b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c5d400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c580d0d40 0x7f4c580cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53330/0 (socket says 172.21.15.105:53330) 2024-03-23T07:56:17.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.418+0000 7f4c5d400640 1 -- 172.21.15.105:0/1895087702 learned_addr learned my addr 172.21.15.105:0/1895087702 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:17.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.419+0000 7f4c5d400640 1 -- 172.21.15.105:0/1895087702 >> v2:172.21.15.132:3300/0 conn(0x7f4c580cd740 msgr2=0x7f4c580b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:17.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.419+0000 7f4c5d400640 1 --2- 172.21.15.105:0/1895087702 >> v2:172.21.15.132:3300/0 conn(0x7f4c580cd740 0x7f4c580b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.419+0000 7f4c5d400640 1 -- 172.21.15.105:0/1895087702 >> v1:172.21.15.132:6789/0 conn(0x7f4c5806b0a0 legacy=0x7f4c5806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.419+0000 7f4c5d400640 1 -- 172.21.15.105:0/1895087702 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580d1790 con 0x7f4c580d0d40 2024-03-23T07:56:17.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.422+0000 7f4c5d400640 1 --2- 172.21.15.105:0/1895087702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580cdb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c5808a100 tx=0x7f4c4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d3cf2ce63f9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:17.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.422+0000 7f4c5ca00640 1 -- 172.21.15.105:0/1895087702 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48030860 con 0x7f4c580d0d40 2024-03-23T07:56:17.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.422+0000 7f4c5ca00640 1 -- 172.21.15.105:0/1895087702 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c480309c0 con 0x7f4c580d0d40 2024-03-23T07:56:17.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.422+0000 7f4c5ca00640 1 -- 172.21.15.105:0/1895087702 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48030d50 con 0x7f4c580d0d40 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 -- 172.21.15.105:0/1895087702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 msgr2=0x7f4c580cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 --2- 172.21.15.105:0/1895087702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580cdb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c5808a100 tx=0x7f4c4802b9a0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 -- 172.21.15.105:0/1895087702 shutdown_connections 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 --2- 172.21.15.105:0/1895087702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580cdb20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 --2- 172.21.15.105:0/1895087702 >> v2:172.21.15.132:3300/0 conn(0x7f4c580cd740 0x7f4c580b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 -- 172.21.15.105:0/1895087702 >> 172.21.15.105:0/1895087702 conn(0x7f4c580cf5d0 msgr2=0x7f4c580cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 -- 172.21.15.105:0/1895087702 shutdown_connections 2024-03-23T07:56:17.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.423+0000 7f4c60800640 1 -- 172.21.15.105:0/1895087702 wait complete. 2024-03-23T07:56:17.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.425+0000 7f4c60800640 1 Processor -- start 2024-03-23T07:56:17.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.425+0000 7f4c60800640 1 -- start start 2024-03-23T07:56:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c60800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5806b0a0 0x7f4c5814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c60800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c580cd740 0x7f4c580ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c60800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c60800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5814fb90 con 0x7f4c580d0d40 2024-03-23T07:56:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c60800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580cb050 con 0x7f4c5806b0a0 2024-03-23T07:56:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c60800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580cb330 con 0x7f4c580cd740 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c5e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c5e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580ace80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53336/0 (socket says 172.21.15.105:53336) 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c5e800640 1 -- 172.21.15.105:0/839105937 learned_addr learned my addr 172.21.15.105:0/839105937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c5de00640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5806b0a0 0x7f4c5814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.426+0000 7f4c5d400640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c580cd740 0x7f4c580ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5e800640 1 -- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c580cd740 msgr2=0x7f4c580ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5e800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c580cd740 0x7f4c580ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5e800640 1 -- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5806b0a0 msgr2=0x7f4c5814f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5e800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5806b0a0 0x7f4c5814f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5e800640 1 -- 172.21.15.105:0/839105937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480094d0 con 0x7f4c580d0d40 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5d400640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c580cd740 0x7f4c580ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:17.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5de00640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5806b0a0 0x7f4c5814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c5e800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580ace80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c7a0 tx=0x7f4c5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c52000640 1 -- 172.21.15.105:0/839105937 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54003b70 con 0x7f4c580d0d40 2024-03-23T07:56:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c52000640 1 -- 172.21.15.105:0/839105937 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54003cf0 con 0x7f4c580d0d40 2024-03-23T07:56:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c52000640 1 -- 172.21.15.105:0/839105937 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54003ea0 con 0x7f4c580d0d40 2024-03-23T07:56:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c580cb610 con 0x7f4c580d0d40 2024-03-23T07:56:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.427+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c580cbbd0 con 0x7f4c580d0d40 2024-03-23T07:56:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.429+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c58066500 con 0x7f4c580d0d40 2024-03-23T07:56:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.430+0000 7f4c52000640 1 -- 172.21.15.105:0/839105937 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4c5400aa90 con 0x7f4c580d0d40 2024-03-23T07:56:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.430+0000 7f4c52000640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4c3c06ac00 0x7f4c3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.430+0000 7f4c52000640 1 -- 172.21.15.105:0/839105937 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4c54089200 con 0x7f4c580d0d40 2024-03-23T07:56:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.431+0000 7f4c5de00640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4c3c06ac00 0x7f4c3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.431+0000 7f4c5de00640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4c3c06ac00 0x7f4c3c06d0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009ac0 tx=0x7f4c4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:17.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.436+0000 7f4c52000640 1 -- 172.21.15.105:0/839105937 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4c540543a0 con 0x7f4c580d0d40 2024-03-23T07:56:17.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.693+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4c58073390 con 0x7f4c580d0d40 2024-03-23T07:56:17.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.703+0000 7f4c52000640 1 -- 172.21.15.105:0/839105937 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4c540579c0 con 0x7f4c580d0d40 2024-03-23T07:56:17.709 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 2 and data pool 3 2024-03-23T07:56:17.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4c3c06ac00 msgr2=0x7f4c3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4c3c06ac00 0x7f4c3c06d0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009ac0 tx=0x7f4c4c008040 comp rx=0 tx=0).stop 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 msgr2=0x7f4c580ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580ace80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c7a0 tx=0x7f4c5400cc60 comp rx=0 tx=0).stop 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 shutdown_connections 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c580d0d40 0x7f4c580ace80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4c3c06ac00 0x7f4c3c06d0c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c580cd740 0x7f4c580ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 --2- 172.21.15.105:0/839105937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5806b0a0 0x7f4c5814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.712+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 >> 172.21.15.105:0/839105937 conn(0x7f4c580cf5d0 msgr2=0x7f4c580c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:17.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.713+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 shutdown_connections 2024-03-23T07:56:17.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.714+0000 7f4c60800640 1 -- 172.21.15.105:0/839105937 wait complete. 2024-03-23T07:56:17.747 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-23T07:56:17.747 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-23T07:56:17.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.925+0000 7f36ef200640 1 Processor -- start 2024-03-23T07:56:17.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.925+0000 7f36ef200640 1 -- start start 2024-03-23T07:56:17.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.926+0000 7f36ef200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f36e815a850 0x7f36e80cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.926+0000 7f36ef200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e815e320 con 0x7f36e80b22b0 2024-03-23T07:56:17.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.926+0000 7f36ef200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80cda20 con 0x7f36e80cacf0 2024-03-23T07:56:17.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.926+0000 7f36ef200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e815ac30 con 0x7f36e815a850 2024-03-23T07:56:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.927+0000 7f36ed400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f36e80b22b0 0x7f36e815eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58482/0 (socket says 172.21.15.105:58482) 2024-03-23T07:56:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.927+0000 7f36ed400640 1 -- 172.21.15.105:0/1423631615 learned_addr learned my addr 172.21.15.105:0/1423631615 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.927+0000 7f36ede00640 1 --2- 172.21.15.105:0/1423631615 >> v2:172.21.15.132:3300/0 conn(0x7f36e815a850 0x7f36e80cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.927+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083988470 0 0) 0x7f36e815e320 con 0x7f36e80b22b0 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.927+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d0003520 con 0x7f36e80b22b0 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3060062607 0 0) 0x7f36d0003520 con 0x7f36e80b22b0 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 >> v2:172.21.15.132:3300/0 conn(0x7f36e815a850 msgr2=0x7f36e80cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 --2- 172.21.15.105:0/1423631615 >> v2:172.21.15.132:3300/0 conn(0x7f36e815a850 0x7f36e80cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 >> v1:172.21.15.132:6789/0 conn(0x7f36e80cacf0 legacy=0x7f36e806d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80cdd00 con 0x7f36e80b22b0 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f36d8001670 con 0x7f36e80b22b0 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 >> v1:172.21.15.105:6789/0 conn(0x7f36e80b22b0 legacy=0x7f36e815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36d0003970 0x7f36d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36d00047c0 0x7f36d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d0005180 0x7f36d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e815ac30 con 0x7f36d00047c0 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c020 con 0x7f36d0005180 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c1e0 con 0x7f36d0003970 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36ed400640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36d00047c0 0x7f36d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36ed400640 1 -- 172.21.15.105:0/1423631615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36d0003970 msgr2=0x7f36d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36ed400640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36d0003970 0x7f36d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36ed400640 1 -- 172.21.15.105:0/1423631615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d0005180 msgr2=0x7f36d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36ee800640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d0005180 0x7f36d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.928+0000 7f36ed400640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d0005180 0x7f36d000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.929+0000 7f36ee800640 1 -- 172.21.15.105:0/1423631615 reap_dead start 2024-03-23T07:56:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.929+0000 7f36ed400640 1 -- 172.21.15.105:0/1423631615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0003520 con 0x7f36d00047c0 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.929+0000 7f36ed400640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36d00047c0 0x7f36d0004c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c190 tx=0x7f36d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34595473d80a7e87 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8005ad0 con 0x7f36d00047c0 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8005c50 con 0x7f36d00047c0 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36eca00640 1 -- 172.21.15.105:0/1423631615 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d802c8a0 con 0x7f36d00047c0 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36ef200640 1 -- 172.21.15.105:0/1423631615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36d00047c0 msgr2=0x7f36d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36ef200640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36d00047c0 0x7f36d0004c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c190 tx=0x7f36d00057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36ef200640 1 -- 172.21.15.105:0/1423631615 shutdown_connections 2024-03-23T07:56:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36ef200640 1 --2- 172.21.15.105:0/1423631615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36d00047c0 0x7f36d0004c40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36ef200640 1 -- 172.21.15.105:0/1423631615 >> 172.21.15.105:0/1423631615 conn(0x7f36e80b01b0 msgr2=0x7f36e80b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36ef200640 1 -- 172.21.15.105:0/1423631615 shutdown_connections 2024-03-23T07:56:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.930+0000 7f36ef200640 1 -- 172.21.15.105:0/1423631615 wait complete. 2024-03-23T07:56:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.932+0000 7f36ef200640 1 Processor -- start 2024-03-23T07:56:17.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.932+0000 7f36ef200640 1 -- start start 2024-03-23T07:56:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ef200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d00047c0 0x7f36e81e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ef200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e8149740 0x7f36e81e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ef200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36e80c9ad0 0x7f36e80c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ef200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80ca490 con 0x7f36e8149740 2024-03-23T07:56:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ef200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81e5e60 con 0x7f36d00047c0 2024-03-23T07:56:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ef200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81e4dc0 con 0x7f36e80c9ad0 2024-03-23T07:56:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ed400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e8149740 0x7f36e81e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ed400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e8149740 0x7f36e81e68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53344/0 (socket says 172.21.15.105:53344) 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.933+0000 7f36ed400640 1 -- 172.21.15.105:0/1348404670 learned_addr learned my addr 172.21.15.105:0/1348404670 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ede00640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d00047c0 0x7f36e81e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ed400640 1 -- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36e80c9ad0 msgr2=0x7f36e80c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ee800640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36e80c9ad0 0x7f36e80c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ed400640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36e80c9ad0 0x7f36e80c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ed400640 1 -- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d00047c0 msgr2=0x7f36e81e47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ed400640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d00047c0 0x7f36e81e47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ed400640 1 -- 172.21.15.105:0/1348404670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e81e50a0 con 0x7f36e8149740 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ede00640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d00047c0 0x7f36e81e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ed400640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e8149740 0x7f36e81e68f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36d8001930 tx=0x7f36d802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36e2000640 1 -- 172.21.15.105:0/1348404670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d803c040 con 0x7f36e8149740 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36e2000640 1 -- 172.21.15.105:0/1348404670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d802cd70 con 0x7f36e8149740 2024-03-23T07:56:17.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36e2000640 1 -- 172.21.15.105:0/1348404670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8007270 con 0x7f36e8149740 2024-03-23T07:56:17.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e81e5380 con 0x7f36e8149740 2024-03-23T07:56:17.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.934+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81e5940 con 0x7f36e8149740 2024-03-23T07:56:17.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.935+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e804e030 con 0x7f36e8149740 2024-03-23T07:56:17.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.936+0000 7f36e2000640 1 -- 172.21.15.105:0/1348404670 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f36d8032070 con 0x7f36e8149740 2024-03-23T07:56:17.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.937+0000 7f36e2000640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36cc06ac00 0x7f36cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:17.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.937+0000 7f36e2000640 1 -- 172.21.15.105:0/1348404670 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36d80aa930 con 0x7f36e8149740 2024-03-23T07:56:17.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.937+0000 7f36ede00640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36cc06ac00 0x7f36cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:17.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.938+0000 7f36ede00640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36cc06ac00 0x7f36cc06d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009aa0 tx=0x7f36dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:17.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:17.941+0000 7f36e2000640 1 -- 172.21.15.105:0/1348404670 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f36d8075a70 con 0x7f36e8149740 2024-03-23T07:56:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.144+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f36e80ccaa0 con 0x7f36e8149740 2024-03-23T07:56:18.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.709+0000 7f36e2000640 1 -- 172.21.15.105:0/1348404670 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) v1 ==== 157+0+0 (secure 0 0 0) 0x7f36d8079090 con 0x7f36e8149740 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36cc06ac00 msgr2=0x7f36cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36cc06ac00 0x7f36cc06d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009aa0 tx=0x7f36dc008040 comp rx=0 tx=0).stop 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e8149740 msgr2=0x7f36e81e68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e8149740 0x7f36e81e68f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36d8001930 tx=0x7f36d802e070 comp rx=0 tx=0).stop 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 shutdown_connections 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36cc06ac00 0x7f36cc06d0c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36e80c9ad0 0x7f36e80c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e8149740 0x7f36e81e68f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 --2- 172.21.15.105:0/1348404670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36d00047c0 0x7f36e81e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 >> 172.21.15.105:0/1348404670 conn(0x7f36e80b01b0 msgr2=0x7f36e80cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 shutdown_connections 2024-03-23T07:56:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.713+0000 7f36ef200640 1 -- 172.21.15.105:0/1348404670 wait complete. 2024-03-23T07:56:18.731 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T07:56:18.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.946+0000 7fe2b2800640 1 Processor -- start 2024-03-23T07:56:18.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.946+0000 7fe2b2800640 1 -- start start 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2b2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac0b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2b2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2ac162020 0x7fe2ac0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2b2800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2ac164940 0x7fe2ac164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2b2800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac165930 con 0x7fe2ac164940 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2b2800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac161350 con 0x7fe2ac162020 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2b2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac147580 con 0x7fe2ac1471a0 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2abe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2ac164940 0x7fe2ac164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2abe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2ac164940 0x7fe2ac164db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53358/0 (socket says 172.21.15.105:53358) 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2abe00640 1 -- 172.21.15.105:0/2792724401 learned_addr learned my addr 172.21.15.105:0/2792724401 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:18.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2ab400640 1 --2- 172.21.15.105:0/2792724401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac0b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.948+0000 7fe2aaa00640 1 --2- 172.21.15.105:0/2792724401 >> v2:172.21.15.132:3300/0 conn(0x7fe2ac162020 0x7fe2ac0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.949+0000 7fe2abe00640 1 -- 172.21.15.105:0/2792724401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 msgr2=0x7fe2ac0b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.949+0000 7fe2abe00640 1 --2- 172.21.15.105:0/2792724401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac0b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.949+0000 7fe2abe00640 1 -- 172.21.15.105:0/2792724401 >> v2:172.21.15.132:3300/0 conn(0x7fe2ac162020 msgr2=0x7fe2ac0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.949+0000 7fe2abe00640 1 --2- 172.21.15.105:0/2792724401 >> v2:172.21.15.132:3300/0 conn(0x7fe2ac162020 0x7fe2ac0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.949+0000 7fe2abe00640 1 -- 172.21.15.105:0/2792724401 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac165650 con 0x7fe2ac164940 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.950+0000 7fe2abe00640 1 --2- 172.21.15.105:0/2792724401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac164db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac06e2e0 tx=0x7fe2a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be991bbe71775143 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.950+0000 7fe2aa000640 1 -- 172.21.15.105:0/2792724401 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a402ca70 con 0x7fe2ac164940 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.950+0000 7fe2aa000640 1 -- 172.21.15.105:0/2792724401 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a402cbd0 con 0x7fe2ac164940 2024-03-23T07:56:18.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.950+0000 7fe2aa000640 1 -- 172.21.15.105:0/2792724401 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a40343f0 con 0x7fe2ac164940 2024-03-23T07:56:18.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 -- 172.21.15.105:0/2792724401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 msgr2=0x7fe2ac164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:18.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 --2- 172.21.15.105:0/2792724401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac164db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac06e2e0 tx=0x7fe2a402bed0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 -- 172.21.15.105:0/2792724401 shutdown_connections 2024-03-23T07:56:18.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 --2- 172.21.15.105:0/2792724401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac164db0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 --2- 172.21.15.105:0/2792724401 >> v2:172.21.15.132:3300/0 conn(0x7fe2ac162020 0x7fe2ac0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 --2- 172.21.15.105:0/2792724401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac0b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 -- 172.21.15.105:0/2792724401 >> 172.21.15.105:0/2792724401 conn(0x7fe2ac069c90 msgr2=0x7fe2ac06a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:18.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 -- 172.21.15.105:0/2792724401 shutdown_connections 2024-03-23T07:56:18.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.951+0000 7fe2b2800640 1 -- 172.21.15.105:0/2792724401 wait complete. 2024-03-23T07:56:18.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.954+0000 7fe2b2800640 1 Processor -- start 2024-03-23T07:56:18.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.954+0000 7fe2b2800640 1 -- start start 2024-03-23T07:56:18.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2b2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac1f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:18.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2b2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ac162020 0x7fe2ac159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:18.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2b2800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:18.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2b2800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac15c780 con 0x7fe2ac164940 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2b2800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac15a460 con 0x7fe2ac162020 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2b2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac15bb70 con 0x7fe2ac1471a0 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac163b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53366/0 (socket says 172.21.15.105:53366) 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2abe00640 1 -- 172.21.15.105:0/3238142556 learned_addr learned my addr 172.21.15.105:0/3238142556 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.955+0000 7fe2ab400640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac1f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2aaa00640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ac162020 0x7fe2ac159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2abe00640 1 -- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 msgr2=0x7fe2ac1f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2abe00640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac1f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2abe00640 1 -- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ac162020 msgr2=0x7fe2ac159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2abe00640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ac162020 0x7fe2ac159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:18.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2abe00640 1 -- 172.21.15.105:0/3238142556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009b30 con 0x7fe2ac164940 2024-03-23T07:56:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2aaa00640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ac162020 0x7fe2ac159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2abe00640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac163b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4036040 tx=0x7fe2a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2a3400640 1 -- 172.21.15.105:0/3238142556 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4040670 con 0x7fe2ac164940 2024-03-23T07:56:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2a3400640 1 -- 172.21.15.105:0/3238142556 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a40407d0 con 0x7fe2ac164940 2024-03-23T07:56:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2a3400640 1 -- 172.21.15.105:0/3238142556 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4040b80 con 0x7fe2ac164940 2024-03-23T07:56:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.956+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac15be50 con 0x7fe2ac164940 2024-03-23T07:56:18.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.957+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac15c410 con 0x7fe2ac164940 2024-03-23T07:56:18.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.958+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac15c8f0 con 0x7fe2ac164940 2024-03-23T07:56:18.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.958+0000 7fe2a3400640 1 -- 172.21.15.105:0/3238142556 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe2a4040ce0 con 0x7fe2ac164940 2024-03-23T07:56:18.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.959+0000 7fe2a3400640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe28c06ac00 0x7fe28c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:18.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.959+0000 7fe2a3400640 1 -- 172.21.15.105:0/3238142556 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fe2a40ac5c0 con 0x7fe2ac164940 2024-03-23T07:56:18.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.959+0000 7fe2ab400640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe28c06ac00 0x7fe28c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:18.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.960+0000 7fe2ab400640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe28c06ac00 0x7fe28c06d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe29c006fd0 tx=0x7fe29c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:18.964+0000 7fe2a3400640 1 -- 172.21.15.105:0/3238142556 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe2a4077620 con 0x7fe2ac164940 2024-03-23T07:56:19.043 INFO:tasks.ceph.mds.b.smithi132.stderr:2024-03-23T07:56:19.044+0000 7f4598c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T07:56:19.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:19.158+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe2ac073390 con 0x7fe2ac164940 2024-03-23T07:56:20.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.975+0000 7fe2a3400640 1 -- 172.21.15.105:0/3238142556 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) v1 ==== 228+0+0 (secure 0 0 0) 0x7fe2a4009510 con 0x7fe2ac164940 2024-03-23T07:56:20.975 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data_ec' created 2024-03-23T07:56:20.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.981+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe28c06ac00 msgr2=0x7fe28c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:20.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.981+0000 7fe2b2800640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe28c06ac00 0x7fe28c06d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe29c006fd0 tx=0x7fe29c008040 comp rx=0 tx=0).stop 2024-03-23T07:56:20.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.981+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 msgr2=0x7fe2ac163b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:20.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.981+0000 7fe2b2800640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac163b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4036040 tx=0x7fe2a4004a40 comp rx=0 tx=0).stop 2024-03-23T07:56:20.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.984+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 shutdown_connections 2024-03-23T07:56:20.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.984+0000 7fe2b2800640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac164940 0x7fe2ac163b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:20.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.984+0000 7fe2b2800640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ac162020 0x7fe2ac159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:20.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.984+0000 7fe2b2800640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe28c06ac00 0x7fe28c06d0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:20.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.984+0000 7fe2b2800640 1 --2- 172.21.15.105:0/3238142556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ac1471a0 0x7fe2ac1f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:20.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.984+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 >> 172.21.15.105:0/3238142556 conn(0x7fe2ac069c90 msgr2=0x7fe2ac162580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:20.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.984+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 shutdown_connections 2024-03-23T07:56:20.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:20.985+0000 7fe2b2800640 1 -- 172.21.15.105:0/3238142556 wait complete. 2024-03-23T07:56:21.010 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-23T07:56:21.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.200+0000 7fe795a00640 1 Processor -- start 2024-03-23T07:56:21.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.201+0000 7fe795a00640 1 -- start start 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe795a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe790159830 0x7fe7900cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe795a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7900ca7b0 0x7fe7900cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe795a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe7900cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe795a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79006de50 con 0x7fe790159830 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe795a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7900cf300 con 0x7fe7900ca7b0 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe795a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe790159c10 con 0x7fe7900ce930 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe78ea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe790159830 0x7fe7900cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe78ea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe790159830 0x7fe7900cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53372/0 (socket says 172.21.15.105:53372) 2024-03-23T07:56:21.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.202+0000 7fe78ea00640 1 -- 172.21.15.105:0/3922453898 learned_addr learned my addr 172.21.15.105:0/3922453898 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:21.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe787e00640 1 --2- 172.21.15.105:0/3922453898 >> v2:172.21.15.132:3300/0 conn(0x7fe7900ca7b0 0x7fe7900cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:21.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe78ea00640 1 -- 172.21.15.105:0/3922453898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 msgr2=0x7fe7900cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:21.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe78f400640 1 --2- 172.21.15.105:0/3922453898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe7900cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:21.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe78ea00640 1 --2- 172.21.15.105:0/3922453898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe7900cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe78ea00640 1 -- 172.21.15.105:0/3922453898 >> v2:172.21.15.132:3300/0 conn(0x7fe7900ca7b0 msgr2=0x7fe7900cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:21.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe78ea00640 1 --2- 172.21.15.105:0/3922453898 >> v2:172.21.15.132:3300/0 conn(0x7fe7900ca7b0 0x7fe7900cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe78ea00640 1 -- 172.21.15.105:0/3922453898 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900cd840 con 0x7fe790159830 2024-03-23T07:56:21.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.203+0000 7fe787e00640 1 --2- 172.21.15.105:0/3922453898 >> v2:172.21.15.132:3300/0 conn(0x7fe7900ca7b0 0x7fe7900cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.204+0000 7fe78ea00640 1 --2- 172.21.15.105:0/3922453898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe790159830 0x7fe7900cc3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7900ad950 tx=0x7fe78002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa6c4caaaea37ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.204+0000 7fe78e000640 1 -- 172.21.15.105:0/3922453898 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780030860 con 0x7fe790159830 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.204+0000 7fe78e000640 1 -- 172.21.15.105:0/3922453898 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800309c0 con 0x7fe790159830 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.204+0000 7fe78e000640 1 -- 172.21.15.105:0/3922453898 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780030d50 con 0x7fe790159830 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 -- 172.21.15.105:0/3922453898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe790159830 msgr2=0x7fe7900cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 --2- 172.21.15.105:0/3922453898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe790159830 0x7fe7900cc3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7900ad950 tx=0x7fe78002b9a0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 -- 172.21.15.105:0/3922453898 shutdown_connections 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 --2- 172.21.15.105:0/3922453898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe7900cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 --2- 172.21.15.105:0/3922453898 >> v2:172.21.15.132:3300/0 conn(0x7fe7900ca7b0 0x7fe7900cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 --2- 172.21.15.105:0/3922453898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe790159830 0x7fe7900cc3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 -- 172.21.15.105:0/3922453898 >> 172.21.15.105:0/3922453898 conn(0x7fe79006a0b0 msgr2=0x7fe79006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:21.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 -- 172.21.15.105:0/3922453898 shutdown_connections 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.205+0000 7fe795a00640 1 -- 172.21.15.105:0/3922453898 wait complete. 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.207+0000 7fe795a00640 1 Processor -- start 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.207+0000 7fe795a00640 1 -- start start 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe795a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7900ca7b0 0x7fe79014dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe795a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe79014ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe795a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe790159830 0x7fe7900b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe795a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79014da30 con 0x7fe7900ca7b0 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe795a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79014cea0 con 0x7fe790159830 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe795a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79014d140 con 0x7fe7900ce930 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7900ca7b0 0x7fe79014dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7900ca7b0 0x7fe79014dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53388/0 (socket says 172.21.15.105:53388) 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 -- 172.21.15.105:0/652338633 learned_addr learned my addr 172.21.15.105:0/652338633 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:21.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe787e00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe79014ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 -- 172.21.15.105:0/652338633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 msgr2=0x7fe79014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe79014ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 -- 172.21.15.105:0/652338633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe790159830 msgr2=0x7fe7900b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe790159830 0x7fe7900b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.208+0000 7fe78ea00640 1 -- 172.21.15.105:0/652338633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800094d0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.209+0000 7fe78ea00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7900ca7b0 0x7fe79014dec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe78002b9e0 tx=0x7fe780004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.209+0000 7fe786a00640 1 -- 172.21.15.105:0/652338633 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780030850 con 0x7fe7900ca7b0 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.209+0000 7fe786a00640 1 -- 172.21.15.105:0/652338633 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe780039d90 con 0x7fe7900ca7b0 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.209+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7900b1030 con 0x7fe7900ca7b0 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.209+0000 7fe786a00640 1 -- 172.21.15.105:0/652338633 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780043650 con 0x7fe7900ca7b0 2024-03-23T07:56:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.209+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79014d3e0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.210+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7900ccef0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.211+0000 7fe786a00640 1 -- 172.21.15.105:0/652338633 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe78004c3b0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.212+0000 7fe786a00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe76c06ac30 0x7fe76c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.212+0000 7fe786a00640 1 -- 172.21.15.105:0/652338633 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fe7800abcb0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.212+0000 7fe787e00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe76c06ac30 0x7fe76c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.213+0000 7fe787e00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe76c06ac30 0x7fe76c06d0f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7900b03e0 tx=0x7fe778008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:21.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.217+0000 7fe786a00640 1 -- 172.21.15.105:0/652338633 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe7900ccef0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.411+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fe7900ccef0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.984+0000 7fe786a00640 1 -- 172.21.15.105:0/652338633 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe7900ccef0 con 0x7fe7900ca7b0 2024-03-23T07:56:21.985 INFO:teuthology.orchestra.run.smithi105.stderr:set pool 4 allow_ec_overwrites to true 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe76c06ac30 msgr2=0x7fe76c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe76c06ac30 0x7fe76c06d0f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7900b03e0 tx=0x7fe778008040 comp rx=0 tx=0).stop 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7900ca7b0 msgr2=0x7fe79014dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7900ca7b0 0x7fe79014dec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe78002b9e0 tx=0x7fe780004730 comp rx=0 tx=0).stop 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 shutdown_connections 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe790159830 0x7fe7900b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe76c06ac30 0x7fe76c06d0f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7900ce930 0x7fe79014ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 --2- 172.21.15.105:0/652338633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7900ca7b0 0x7fe79014dec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:21.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.990+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 >> 172.21.15.105:0/652338633 conn(0x7fe79006a0b0 msgr2=0x7fe79006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:21.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.997+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 shutdown_connections 2024-03-23T07:56:21.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:21.997+0000 7fe795a00640 1 -- 172.21.15.105:0/652338633 wait complete. 2024-03-23T07:56:22.032 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-23T07:56:22.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.212+0000 7f58ca400640 1 Processor -- start 2024-03-23T07:56:22.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.212+0000 7f58ca400640 1 -- start start 2024-03-23T07:56:22.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.213+0000 7f58ca400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 0x7f58c406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.213+0000 7f58ca400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c40cdba0 con 0x7f58c40ce930 2024-03-23T07:56:22.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.213+0000 7f58ca400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c40cde80 con 0x7f58c40add30 2024-03-23T07:56:22.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.213+0000 7f58ca400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40ae1c0 con 0x7f58c406ce90 2024-03-23T07:56:22.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c3400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f58c40ce930 0x7f58c40cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58484/0 (socket says 172.21.15.105:58484) 2024-03-23T07:56:22.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c3400640 1 -- 172.21.15.105:0/3518113996 learned_addr learned my addr 172.21.15.105:0/3518113996 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398526042 0 0) 0x7f58c40cdba0 con 0x7f58c40ce930 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a8003520 con 0x7f58c40ce930 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c3e00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 0x7f58c406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627383682 0 0) 0x7f58a8003520 con 0x7f58c40ce930 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 msgr2=0x7f58c406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 0x7f58c406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 >> v1:172.21.15.132:6789/0 conn(0x7f58c40add30 legacy=0x7f58c40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40ce160 con 0x7f58c40ce930 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f58b40019f0 con 0x7f58c40ce930 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.214+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 >> v1:172.21.15.105:6789/0 conn(0x7f58c40ce930 legacy=0x7f58c40cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c2a00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a80039a0 0x7f58a8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c2a00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 0x7f58a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c2a00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58a8004d00 0x7f58a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a800bb60 con 0x7f58a8004d00 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a800bca0 con 0x7f58a80039a0 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a800be60 con 0x7f58c406ce90 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c3e00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58a8004d00 0x7f58a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c3400640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a80039a0 0x7f58a8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58baa00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 0x7f58a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c3e00640 1 -- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 msgr2=0x7f58a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c3e00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 0x7f58a80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c3e00640 1 -- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a80039a0 msgr2=0x7f58a8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c3e00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a80039a0 0x7f58a8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58c3e00640 1 -- 172.21.15.105:0/3518113996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a8003520 con 0x7f58a8004d00 2024-03-23T07:56:22.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.215+0000 7f58baa00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406ce90 0x7f58a80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:22.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.216+0000 7f58c3e00640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58a8004d00 0x7f58a8005180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00ba30 tx=0x7f58a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d09547b6b2cb0b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc032630 con 0x7f58a8004d00 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc032790 con 0x7f58a8004d00 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58c2a00640 1 -- 172.21.15.105:0/3518113996 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc032b20 con 0x7f58a8004d00 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58ca400640 1 -- 172.21.15.105:0/3518113996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58a8004d00 msgr2=0x7f58a8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58ca400640 1 --2- 172.21.15.105:0/3518113996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58a8004d00 0x7f58a8005180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00ba30 tx=0x7f58a80057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58c3e00640 1 -- 172.21.15.105:0/3518113996 reap_dead start 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58ca400640 1 -- 172.21.15.105:0/3518113996 shutdown_connections 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58ca400640 1 -- 172.21.15.105:0/3518113996 >> 172.21.15.105:0/3518113996 conn(0x7f58c406a0b0 msgr2=0x7f58c406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58ca400640 1 -- 172.21.15.105:0/3518113996 shutdown_connections 2024-03-23T07:56:22.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.217+0000 7f58ca400640 1 -- 172.21.15.105:0/3518113996 wait complete. 2024-03-23T07:56:22.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.219+0000 7f58ca400640 1 Processor -- start 2024-03-23T07:56:22.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.219+0000 7f58ca400640 1 -- start start 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58ca400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a8004d00 0x7f58c414d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58ca400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58c4149200 0x7f58c40b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58ca400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4149680 0x7f58c40b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58ca400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415b2c0 con 0x7f58c4149200 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58ca400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c414dbe0 con 0x7f58a8004d00 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58ca400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415afe0 con 0x7f58c4149680 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58baa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58c4149200 0x7f58c40b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58baa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58c4149200 0x7f58c40b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53410/0 (socket says 172.21.15.105:53410) 2024-03-23T07:56:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.220+0000 7f58baa00640 1 -- 172.21.15.105:0/3052848288 learned_addr learned my addr 172.21.15.105:0/3052848288 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58c3400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a8004d00 0x7f58c414d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58c3e00640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4149680 0x7f58c40b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58baa00640 1 -- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4149680 msgr2=0x7f58c40b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58baa00640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4149680 0x7f58c40b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58baa00640 1 -- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a8004d00 msgr2=0x7f58c414d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58baa00640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a8004d00 0x7f58c414d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58baa00640 1 -- 172.21.15.105:0/3052848288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c414e4f0 con 0x7f58c4149200 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58c3400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a8004d00 0x7f58c414d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58c3e00640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4149680 0x7f58c40b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58baa00640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58c4149200 0x7f58c40b02e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d770 tx=0x7f58ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58bbe00640 1 -- 172.21.15.105:0/3052848288 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac005270 con 0x7f58c4149200 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58bbe00640 1 -- 172.21.15.105:0/3052848288 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac0053d0 con 0x7f58c4149200 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58bbe00640 1 -- 172.21.15.105:0/3052848288 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00b870 con 0x7f58c4149200 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.221+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c414e7d0 con 0x7f58c4149200 2024-03-23T07:56:22.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.222+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c414bb40 con 0x7f58c4149200 2024-03-23T07:56:22.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.223+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4073390 con 0x7f58c4149200 2024-03-23T07:56:22.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.224+0000 7f58bbe00640 1 -- 172.21.15.105:0/3052848288 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f58ac00b9d0 con 0x7f58c4149200 2024-03-23T07:56:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.225+0000 7f58bbe00640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f58a006ac00 0x7f58a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.225+0000 7f58bbe00640 1 -- 172.21.15.105:0/3052848288 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f58ac089570 con 0x7f58c4149200 2024-03-23T07:56:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.225+0000 7f58c3400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f58a006ac00 0x7f58a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.226+0000 7f58c3400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f58a006ac00 0x7f58a006d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58b4001f40 tx=0x7f58b4001330 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:22.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.230+0000 7f58bbe00640 1 -- 172.21.15.105:0/3052848288 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f58ac054460 con 0x7f58c4149200 2024-03-23T07:56:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.455+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f58c40cef60 con 0x7f58c4149200 2024-03-23T07:56:22.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.467+0000 7f58bbe00640 1 -- 172.21.15.105:0/3052848288 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f58ac019310 con 0x7f58c4149200 2024-03-23T07:56:22.467 INFO:teuthology.orchestra.run.smithi105.stderr:added data pool 4 to fsmap 2024-03-23T07:56:22.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.469+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f58a006ac00 msgr2=0x7f58a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.469+0000 7f58ca400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f58a006ac00 0x7f58a006d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58b4001f40 tx=0x7f58b4001330 comp rx=0 tx=0).stop 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58c4149200 msgr2=0x7f58c40b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58c4149200 0x7f58c40b02e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d770 tx=0x7f58ac00dc30 comp rx=0 tx=0).stop 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 shutdown_connections 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f58a006ac00 0x7f58a006d0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4149680 0x7f58c40b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58c4149200 0x7f58c40b02e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 --2- 172.21.15.105:0/3052848288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a8004d00 0x7f58c414d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 >> 172.21.15.105:0/3052848288 conn(0x7f58c406a0b0 msgr2=0x7f58c40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 shutdown_connections 2024-03-23T07:56:22.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.470+0000 7f58ca400640 1 -- 172.21.15.105:0/3052848288 wait complete. 2024-03-23T07:56:22.486 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:56:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.682+0000 7f403c800640 1 Processor -- start 2024-03-23T07:56:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.682+0000 7f403c800640 1 -- start start 2024-03-23T07:56:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.684+0000 7f403c800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f40340b39b0 0x7f40340b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.684+0000 7f403c800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403406c6f0 con 0x7f403415b560 2024-03-23T07:56:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.684+0000 7f403c800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403406c9d0 con 0x7f403415c400 2024-03-23T07:56:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.684+0000 7f403c800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40340b4370 con 0x7f40340b39b0 2024-03-23T07:56:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.684+0000 7f4039e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f403415b560 0x7f403415b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58486/0 (socket says 172.21.15.105:58486) 2024-03-23T07:56:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.684+0000 7f4039e00640 1 -- 172.21.15.105:0/4025458608 learned_addr learned my addr 172.21.15.105:0/4025458608 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4039400640 1 --2- 172.21.15.105:0/4025458608 >> v2:172.21.15.132:3300/0 conn(0x7f40340b39b0 0x7f40340b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911133378 0 0) 0x7f403406c6f0 con 0x7f403415b560 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f401c003520 con 0x7f403415b560 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1350607034 0 0) 0x7f401c003520 con 0x7f403415b560 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 >> v2:172.21.15.132:3300/0 conn(0x7f40340b39b0 msgr2=0x7f40340b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 --2- 172.21.15.105:0/4025458608 >> v2:172.21.15.132:3300/0 conn(0x7f40340b39b0 0x7f40340b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 >> v1:172.21.15.132:6789/0 conn(0x7f403415c400 legacy=0x7f403415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403406ccb0 con 0x7f403415b560 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f4028001670 con 0x7f403415b560 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 >> v1:172.21.15.105:6789/0 conn(0x7f403415b560 legacy=0x7f403415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f401c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f401c0047c0 0x7f401c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f401c005180 0x7f401c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40340b4370 con 0x7f401c003970 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c00c020 con 0x7f401c0047c0 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.685+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c00c1e0 con 0x7f401c005180 2024-03-23T07:56:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.686+0000 7f4039e00640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f401c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.686+0000 7f4039e00640 1 -- 172.21.15.105:0/4025458608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f401c005180 msgr2=0x7f401c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.686+0000 7f4039e00640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f401c005180 0x7f401c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.686+0000 7f4039e00640 1 -- 172.21.15.105:0/4025458608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f401c0047c0 msgr2=0x7f401c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.686+0000 7f4039e00640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f401c0047c0 0x7f401c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.686+0000 7f4039e00640 1 -- 172.21.15.105:0/4025458608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c003520 con 0x7f401c003970 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.686+0000 7f403a800640 1 -- 172.21.15.105:0/4025458608 reap_dead start 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.687+0000 7f4039e00640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f401c003d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f402800c190 tx=0x7f401c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ad4b07197ce2a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.687+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028005ad0 con 0x7f401c003970 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.687+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028005c30 con 0x7f401c003970 2024-03-23T07:56:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.687+0000 7f4038a00640 1 -- 172.21.15.105:0/4025458608 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402802cae0 con 0x7f401c003970 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.688+0000 7f403c800640 1 -- 172.21.15.105:0/4025458608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 msgr2=0x7f401c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.688+0000 7f403c800640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f401c003d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f402800c190 tx=0x7f401c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.688+0000 7f403c800640 1 -- 172.21.15.105:0/4025458608 shutdown_connections 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.688+0000 7f403c800640 1 --2- 172.21.15.105:0/4025458608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f401c003d90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.688+0000 7f403c800640 1 -- 172.21.15.105:0/4025458608 >> 172.21.15.105:0/4025458608 conn(0x7f403406a3d0 msgr2=0x7f40340b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.688+0000 7f403c800640 1 -- 172.21.15.105:0/4025458608 shutdown_connections 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.688+0000 7f403c800640 1 -- 172.21.15.105:0/4025458608 wait complete. 2024-03-23T07:56:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.690+0000 7f403c800640 1 Processor -- start 2024-03-23T07:56:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.690+0000 7f403c800640 1 -- start start 2024-03-23T07:56:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.690+0000 7f403c800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f40340cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.690+0000 7f403c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40341f2d50 0x7f40341f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f403c800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40341411b0 0x7f4034141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f403c800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4034141b70 con 0x7f401c003970 2024-03-23T07:56:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f403c800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40341f31b0 con 0x7f40341411b0 2024-03-23T07:56:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f403c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40340cbf20 con 0x7f40341f2d50 2024-03-23T07:56:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f40340cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f40340cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53432/0 (socket says 172.21.15.105:53432) 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 -- 172.21.15.105:0/2986988287 learned_addr learned my addr 172.21.15.105:0/2986988287 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 -- 172.21.15.105:0/2986988287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40341f2d50 msgr2=0x7f40341f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40341f2d50 0x7f40341f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 -- 172.21.15.105:0/2986988287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40341411b0 msgr2=0x7f4034141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40341411b0 0x7f4034141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 -- 172.21.15.105:0/2986988287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40340cc200 con 0x7f401c003970 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.691+0000 7f4039e00640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f40340cb8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4028001930 tx=0x7f402802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.692+0000 7f402e000640 1 -- 172.21.15.105:0/2986988287 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402803c040 con 0x7f401c003970 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.692+0000 7f402e000640 1 -- 172.21.15.105:0/2986988287 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402802cdf0 con 0x7f401c003970 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.692+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40340cc4e0 con 0x7f401c003970 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.692+0000 7f402e000640 1 -- 172.21.15.105:0/2986988287 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280072b0 con 0x7f401c003970 2024-03-23T07:56:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.692+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40340ccaa0 con 0x7f401c003970 2024-03-23T07:56:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.693+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403406ac20 con 0x7f401c003970 2024-03-23T07:56:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.694+0000 7f402e000640 1 -- 172.21.15.105:0/2986988287 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4028032070 con 0x7f401c003970 2024-03-23T07:56:22.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.695+0000 7f402e000640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f401806ac00 0x7f401806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:22.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.695+0000 7f402e000640 1 -- 172.21.15.105:0/2986988287 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f40280aaad0 con 0x7f401c003970 2024-03-23T07:56:22.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.696+0000 7f4039400640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f401806ac00 0x7f401806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:22.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.696+0000 7f4039400640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f401806ac00 0x7f401806d0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40240093d0 tx=0x7f4024009300 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.700+0000 7f402e000640 1 -- 172.21.15.105:0/2986988287 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4028075a50 con 0x7f401c003970 2024-03-23T07:56:22.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.930+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4034073390 con 0x7f401c003970 2024-03-23T07:56:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.930+0000 7f402e000640 1 -- 172.21.15.105:0/2986988287 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4739 (secure 0 0 0) 0x7f4028079070 con 0x7f401c003970 2024-03-23T07:56:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:22.930 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:22.457045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T07:56:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f401806ac00 msgr2=0x7f401806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f401806ac00 0x7f401806d0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40240093d0 tx=0x7f4024009300 comp rx=0 tx=0).stop 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 msgr2=0x7f40340cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f40340cb8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4028001930 tx=0x7f402802e070 comp rx=0 tx=0).stop 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 shutdown_connections 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40341411b0 0x7f4034141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f401806ac00 0x7f401806d0c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40341f2d50 0x7f40341f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 --2- 172.21.15.105:0/2986988287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c003970 0x7f40340cb8b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 >> 172.21.15.105:0/2986988287 conn(0x7f403406a3d0 msgr2=0x7f403415d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 shutdown_connections 2024-03-23T07:56:22.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:22.933+0000 7f403c800640 1 -- 172.21.15.105:0/2986988287 wait complete. 2024-03-23T07:56:22.950 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:23.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.157+0000 7fdef5c00640 1 Processor -- start 2024-03-23T07:56:23.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.157+0000 7fdef5c00640 1 -- start start 2024-03-23T07:56:23.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.158+0000 7fdef5c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdef00adbc0 0x7fdef00b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.158+0000 7fdef5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.158+0000 7fdef5c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef00b16a0 con 0x7fdef006d390 2024-03-23T07:56:23.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.158+0000 7fdef5c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef00b4700 con 0x7fdef00adbc0 2024-03-23T07:56:23.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.158+0000 7fdef5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef00cbfb0 con 0x7fdef00cbab0 2024-03-23T07:56:23.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.159+0000 7fdee6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.159+0000 7fdeeea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdef00adbc0 0x7fdef00b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.159+0000 7fdeeea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdef00adbc0 0x7fdef00b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:56244/0 (socket says 172.21.15.105:56244) 2024-03-23T07:56:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.159+0000 7fdeeea00640 1 -- 172.21.15.105:0/2434813016 learned_addr learned my addr 172.21.15.105:0/2434813016 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.159+0000 7fdeef400640 1 --1- 172.21.15.105:0/2434813016 >> v1:172.21.15.132:6789/0 conn(0x7fdef006d390 0x7fdef00b2aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.105:41298/0 (socket says 172.21.15.105:41298) 2024-03-23T07:56:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.160+0000 7fdeeea00640 1 -- 172.21.15.105:0/2434813016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 msgr2=0x7fdef00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.160+0000 7fdeeea00640 1 --2- 172.21.15.105:0/2434813016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.160+0000 7fdeeea00640 1 -- 172.21.15.105:0/2434813016 >> v1:172.21.15.132:6789/0 conn(0x7fdef006d390 legacy=0x7fdef00b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.160+0000 7fdeeea00640 1 -- 172.21.15.105:0/2434813016 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00cc290 con 0x7fdef00adbc0 2024-03-23T07:56:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.161+0000 7fdeeea00640 1 --2- 172.21.15.105:0/2434813016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef00b41c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009870 tx=0x7fdee002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4623ee3a294bd94f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:23.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdeee000640 1 -- 172.21.15.105:0/2434813016 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0031640 con 0x7fdef00adbc0 2024-03-23T07:56:23.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdeee000640 1 -- 172.21.15.105:0/2434813016 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee00317a0 con 0x7fdef00adbc0 2024-03-23T07:56:23.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdeee000640 1 -- 172.21.15.105:0/2434813016 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0031b30 con 0x7fdef00adbc0 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 -- 172.21.15.105:0/2434813016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 msgr2=0x7fdef00b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2434813016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef00b41c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009870 tx=0x7fdee002b8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 -- 172.21.15.105:0/2434813016 shutdown_connections 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2434813016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2434813016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef00b41c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 -- 172.21.15.105:0/2434813016 >> 172.21.15.105:0/2434813016 conn(0x7fdef006a3d0 msgr2=0x7fdef00b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 -- 172.21.15.105:0/2434813016 shutdown_connections 2024-03-23T07:56:23.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.162+0000 7fdef5c00640 1 -- 172.21.15.105:0/2434813016 wait complete. 2024-03-23T07:56:23.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.164+0000 7fdef5c00640 1 Processor -- start 2024-03-23T07:56:23.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.164+0000 7fdef5c00640 1 -- start start 2024-03-23T07:56:23.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdef5c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdef006d390 0x7fdef0159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdef5c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef0153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdef5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdef5c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef015ce80 con 0x7fdef006d390 2024-03-23T07:56:23.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdef5c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef013b3c0 con 0x7fdef00adbc0 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdef5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef013c1e0 con 0x7fdef00cbab0 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdeeea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdef006d390 0x7fdef0159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdeeea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdef006d390 0x7fdef0159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53436/0 (socket says 172.21.15.105:53436) 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.165+0000 7fdeeea00640 1 -- 172.21.15.105:0/2469828119 learned_addr learned my addr 172.21.15.105:0/2469828119 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeef400640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdee6000640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef0153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeeea00640 1 -- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 msgr2=0x7fdef015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeeea00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeeea00640 1 -- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 msgr2=0x7fdef0153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeeea00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef0153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeeea00640 1 -- 172.21.15.105:0/2469828119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00094d0 con 0x7fdef006d390 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdee6000640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef0153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:23.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeef400640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:23.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdeeea00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdef006d390 0x7fdef0159610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdee0000c00 tx=0x7fdee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:23.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.166+0000 7fdee7400640 1 -- 172.21.15.105:0/2469828119 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee003a930 con 0x7fdef006d390 2024-03-23T07:56:23.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.167+0000 7fdee7400640 1 -- 172.21.15.105:0/2469828119 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee003aa90 con 0x7fdef006d390 2024-03-23T07:56:23.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.167+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef013b8b0 con 0x7fdef006d390 2024-03-23T07:56:23.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.167+0000 7fdee7400640 1 -- 172.21.15.105:0/2469828119 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee003ae40 con 0x7fdef006d390 2024-03-23T07:56:23.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.167+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef013bb90 con 0x7fdef006d390 2024-03-23T07:56:23.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.168+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef006ac20 con 0x7fdef006d390 2024-03-23T07:56:23.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.169+0000 7fdee7400640 1 -- 172.21.15.105:0/2469828119 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdee002f050 con 0x7fdef006d390 2024-03-23T07:56:23.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.169+0000 7fdee7400640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdecc06ac30 0x7fdecc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.170+0000 7fdee7400640 1 -- 172.21.15.105:0/2469828119 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdee00ab460 con 0x7fdef006d390 2024-03-23T07:56:23.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.170+0000 7fdee6000640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdecc06ac30 0x7fdecc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.170+0000 7fdee6000640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdecc06ac30 0x7fdecc06d0f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdef00b0370 tx=0x7fded8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.174+0000 7fdee7400640 1 -- 172.21.15.105:0/2469828119 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdee00762b0 con 0x7fdef006d390 2024-03-23T07:56:23.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.368+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdef0073390 con 0x7fdef006d390 2024-03-23T07:56:23.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.371+0000 7fdee7400640 1 -- 172.21.15.105:0/2469828119 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7fdee00798d0 con 0x7fdef006d390 2024-03-23T07:56:23.372 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:23.372 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:23.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdecc06ac30 msgr2=0x7fdecc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdecc06ac30 0x7fdecc06d0f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdef00b0370 tx=0x7fded8008040 comp rx=0 tx=0).stop 2024-03-23T07:56:23.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdef006d390 msgr2=0x7fdef0159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdef006d390 0x7fdef0159610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdee0000c00 tx=0x7fdee00046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 shutdown_connections 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdecc06ac30 0x7fdecc06d0f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdef00cbab0 0x7fdef015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdef00adbc0 0x7fdef0153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 --2- 172.21.15.105:0/2469828119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdef006d390 0x7fdef0159610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.374+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 >> 172.21.15.105:0/2469828119 conn(0x7fdef006a3d0 msgr2=0x7fdef0142bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.375+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 shutdown_connections 2024-03-23T07:56:23.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.375+0000 7fdef5c00640 1 -- 172.21.15.105:0/2469828119 wait complete. 2024-03-23T07:56:23.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:23.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.592+0000 7fa577200640 1 Processor -- start 2024-03-23T07:56:23.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.593+0000 7fa577200640 1 -- start start 2024-03-23T07:56:23.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.594+0000 7fa577200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5700b3e30 0x7fa57015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.594+0000 7fa577200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57006c6f0 con 0x7fa57015b560 2024-03-23T07:56:23.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.594+0000 7fa577200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57006c9d0 con 0x7fa5700b39b0 2024-03-23T07:56:23.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.594+0000 7fa577200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57006ccb0 con 0x7fa5700b3e30 2024-03-23T07:56:23.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.594+0000 7fa56fe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa57015b560 0x7fa57015b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58498/0 (socket says 172.21.15.105:58498) 2024-03-23T07:56:23.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.594+0000 7fa56fe00640 1 -- 172.21.15.105:0/1208656105 learned_addr learned my addr 172.21.15.105:0/1208656105 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365178706 0 0) 0x7fa57006c6f0 con 0x7fa57015b560 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa554003520 con 0x7fa57015b560 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa575200640 1 --2- 172.21.15.105:0/1208656105 >> v2:172.21.15.132:3300/0 conn(0x7fa5700b3e30 0x7fa57015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681202109 0 0) 0x7fa554003520 con 0x7fa57015b560 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 >> v2:172.21.15.132:3300/0 conn(0x7fa5700b3e30 msgr2=0x7fa57015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 --2- 172.21.15.105:0/1208656105 >> v2:172.21.15.132:3300/0 conn(0x7fa5700b3e30 0x7fa57015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 >> v1:172.21.15.132:6789/0 conn(0x7fa5700b39b0 legacy=0x7fa57015c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57006cf90 con 0x7fa57015b560 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fa560001670 con 0x7fa57015b560 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 >> v1:172.21.15.105:6789/0 conn(0x7fa57015b560 legacy=0x7fa57015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5540039c0 0x7fa554003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5540047c0 0x7fa554004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa554005380 0x7fa55400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57006ccb0 con 0x7fa5540039c0 2024-03-23T07:56:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55400c1f0 con 0x7fa554005380 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55400c3b0 con 0x7fa5540047c0 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.595+0000 7fa575200640 1 --2- 172.21.15.105:0/1208656105 >> v2:172.21.15.132:3300/0 conn(0x7fa5700b3e30 0x7fa57015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa56fe00640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5540039c0 0x7fa554003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa567400640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5540047c0 0x7fa554004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa56fe00640 1 -- 172.21.15.105:0/1208656105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5540047c0 msgr2=0x7fa554004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa56fe00640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5540047c0 0x7fa554004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa56fe00640 1 -- 172.21.15.105:0/1208656105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa554005380 msgr2=0x7fa55400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa56fe00640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa554005380 0x7fa55400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa56fe00640 1 -- 172.21.15.105:0/1208656105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa554003520 con 0x7fa5540039c0 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.596+0000 7fa575200640 1 -- 172.21.15.105:0/1208656105 reap_dead start 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.597+0000 7fa56fe00640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5540039c0 0x7fa554003de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c0e0 tx=0x7fa5540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7104dc9e287f56c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.597+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560005ad0 con 0x7fa5540039c0 2024-03-23T07:56:23.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.597+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa560005c30 con 0x7fa5540039c0 2024-03-23T07:56:23.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.597+0000 7fa56f400640 1 -- 172.21.15.105:0/1208656105 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56002c7e0 con 0x7fa5540039c0 2024-03-23T07:56:23.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.598+0000 7fa577200640 1 -- 172.21.15.105:0/1208656105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5540039c0 msgr2=0x7fa554003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.598+0000 7fa577200640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5540039c0 0x7fa554003de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c0e0 tx=0x7fa5540057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.598+0000 7fa577200640 1 -- 172.21.15.105:0/1208656105 shutdown_connections 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.598+0000 7fa577200640 1 --2- 172.21.15.105:0/1208656105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5540039c0 0x7fa554003de0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.598+0000 7fa577200640 1 -- 172.21.15.105:0/1208656105 >> 172.21.15.105:0/1208656105 conn(0x7fa57006a3d0 msgr2=0x7fa5700b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.598+0000 7fa577200640 1 -- 172.21.15.105:0/1208656105 shutdown_connections 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.598+0000 7fa577200640 1 -- 172.21.15.105:0/1208656105 wait complete. 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.599+0000 7fa577200640 1 Processor -- start 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.600+0000 7fa577200640 1 -- start start 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.600+0000 7fa577200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5540039c0 0x7fa570151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.600+0000 7fa577200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa570156920 0x7fa570152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa577200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa570156da0 0x7fa570153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa577200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701517d0 con 0x7fa570156920 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa577200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700ae490 con 0x7fa5540039c0 2024-03-23T07:56:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa577200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700ae770 con 0x7fa570156da0 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa570156920 0x7fa570152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa570156920 0x7fa570152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53442/0 (socket says 172.21.15.105:53442) 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 -- 172.21.15.105:0/2379169178 learned_addr learned my addr 172.21.15.105:0/2379169178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 -- 172.21.15.105:0/2379169178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa570156da0 msgr2=0x7fa570153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa570156da0 0x7fa570153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 -- 172.21.15.105:0/2379169178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5540039c0 msgr2=0x7fa570151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa56fe00640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5540039c0 0x7fa570151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5540039c0 0x7fa570151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 -- 172.21.15.105:0/2379169178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5700aea50 con 0x7fa570156920 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.601+0000 7fa567400640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa570156920 0x7fa570152c70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa55800d780 tx=0x7fa55800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.602+0000 7fa56cc00640 1 -- 172.21.15.105:0/2379169178 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558003da0 con 0x7fa570156920 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.602+0000 7fa56cc00640 1 -- 172.21.15.105:0/2379169178 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55800de90 con 0x7fa570156920 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.602+0000 7fa56cc00640 1 -- 172.21.15.105:0/2379169178 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55800b8d0 con 0x7fa570156920 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.602+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5700aed30 con 0x7fa570156920 2024-03-23T07:56:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.602+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5701531b0 con 0x7fa570156920 2024-03-23T07:56:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.603+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5700ae5d0 con 0x7fa570156920 2024-03-23T07:56:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.604+0000 7fa56cc00640 1 -- 172.21.15.105:0/2379169178 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa55800ba30 con 0x7fa570156920 2024-03-23T07:56:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.605+0000 7fa56cc00640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa54c06ac00 0x7fa54c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.605+0000 7fa56cc00640 1 -- 172.21.15.105:0/2379169178 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa5580897a0 con 0x7fa570156920 2024-03-23T07:56:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.605+0000 7fa56fe00640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa54c06ac00 0x7fa54c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.606+0000 7fa56fe00640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa54c06ac00 0x7fa54c06d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa560034000 tx=0x7fa560009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.609+0000 7fa56cc00640 1 -- 172.21.15.105:0/2379169178 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa5700ae5d0 con 0x7fa570156920 2024-03-23T07:56:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.803+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5700ae5d0 con 0x7fa570156920 2024-03-23T07:56:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.806+0000 7fa56cc00640 1 -- 172.21.15.105:0/2379169178 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7fa5700ae5d0 con 0x7fa570156920 2024-03-23T07:56:23.806 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:23.806 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa54c06ac00 msgr2=0x7fa54c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa54c06ac00 0x7fa54c06d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa560034000 tx=0x7fa560009030 comp rx=0 tx=0).stop 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa570156920 msgr2=0x7fa570152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa570156920 0x7fa570152c70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa55800d780 tx=0x7fa55800dc40 comp rx=0 tx=0).stop 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 shutdown_connections 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa54c06ac00 0x7fa54c06d0c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa570156da0 0x7fa570153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa570156920 0x7fa570152c70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 --2- 172.21.15.105:0/2379169178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5540039c0 0x7fa570151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 >> 172.21.15.105:0/2379169178 conn(0x7fa57006a3d0 msgr2=0x7fa57015c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 shutdown_connections 2024-03-23T07:56:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:23.809+0000 7fa577200640 1 -- 172.21.15.105:0/2379169178 wait complete. 2024-03-23T07:56:23.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:56:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.049+0000 7fc3ab200640 1 Processor -- start 2024-03-23T07:56:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.049+0000 7fc3ab200640 1 -- start start 2024-03-23T07:56:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3ab200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc3ac06a3a0 0x7fc3ac0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3ab200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ac06aa30 con 0x7fc3ac0ada70 2024-03-23T07:56:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3ab200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ac0b3f40 con 0x7fc3ac0b08c0 2024-03-23T07:56:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3ab200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ac15cee0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3a9e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc3ac0ada70 0x7fc3ac15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58514/0 (socket says 172.21.15.105:58514) 2024-03-23T07:56:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3a9e00640 1 -- 172.21.15.105:0/2878890736 learned_addr learned my addr 172.21.15.105:0/2878890736 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3aa800640 1 --2- 172.21.15.105:0/2878890736 >> v2:172.21.15.105:3300/0 conn(0x7fc3ac06a3a0 0x7fc3ac0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.051+0000 7fc3a8a00640 1 -- 172.21.15.105:0/2878890736 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078514445 0 0) 0x7fc3ac06aa30 con 0x7fc3ac0ada70 2024-03-23T07:56:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.052+0000 7fc3a8a00640 1 -- 172.21.15.105:0/2878890736 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc390003520 con 0x7fc3ac0ada70 2024-03-23T07:56:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.052+0000 7fc3aa800640 1 -- 172.21.15.105:0/2878890736 >> v1:172.21.15.132:6789/0 conn(0x7fc3ac0b08c0 legacy=0x7fc3ac069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.052+0000 7fc3aa800640 1 -- 172.21.15.105:0/2878890736 >> v1:172.21.15.105:6789/0 conn(0x7fc3ac0ada70 legacy=0x7fc3ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.052+0000 7fc3aa800640 1 -- 172.21.15.105:0/2878890736 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac0b41c0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.053+0000 7fc3aa800640 1 --2- 172.21.15.105:0/2878890736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac0b23c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009d90 tx=0x7fc3a402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3c98eb17c826ec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.053+0000 7fc3a8a00640 1 -- 172.21.15.105:0/2878890736 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a402cc60 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.053+0000 7fc3a8a00640 1 -- 172.21.15.105:0/2878890736 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a402cdc0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.053+0000 7fc3a8a00640 1 -- 172.21.15.105:0/2878890736 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4034660 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.054+0000 7fc3ab200640 1 -- 172.21.15.105:0/2878890736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 msgr2=0x7fc3ac0b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.054+0000 7fc3ab200640 1 --2- 172.21.15.105:0/2878890736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac0b23c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009d90 tx=0x7fc3a402beb0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.054+0000 7fc3ab200640 1 -- 172.21.15.105:0/2878890736 shutdown_connections 2024-03-23T07:56:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.054+0000 7fc3ab200640 1 --2- 172.21.15.105:0/2878890736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac0b23c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.054+0000 7fc3ab200640 1 -- 172.21.15.105:0/2878890736 >> 172.21.15.105:0/2878890736 conn(0x7fc3ac0b48e0 msgr2=0x7fc3ac0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.054+0000 7fc3ab200640 1 -- 172.21.15.105:0/2878890736 shutdown_connections 2024-03-23T07:56:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.054+0000 7fc3ab200640 1 -- 172.21.15.105:0/2878890736 wait complete. 2024-03-23T07:56:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.056+0000 7fc3ab200640 1 Processor -- start 2024-03-23T07:56:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.056+0000 7fc3ab200640 1 -- start start 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3ab200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac1f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3ab200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3ac0ada70 0x7fc3ac0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3ab200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3ac0b08c0 0x7fc3ac1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3ab200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1f58c0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3ab200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1f21d0 con 0x7fc3ac0b08c0 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3ab200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac0aeeb0 con 0x7fc3ac0ada70 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac1f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac1f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53470/0 (socket says 172.21.15.105:53470) 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 -- 172.21.15.105:0/1007700656 learned_addr learned my addr 172.21.15.105:0/1007700656 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3aa800640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3ac0b08c0 0x7fc3ac1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 -- 172.21.15.105:0/1007700656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3ac0ada70 msgr2=0x7fc3ac0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3ac0ada70 0x7fc3ac0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 -- 172.21.15.105:0/1007700656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3ac0b08c0 msgr2=0x7fc3ac1536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3ac0b08c0 0x7fc3ac1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.057+0000 7fc3a9e00640 1 -- 172.21.15.105:0/1007700656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4009a70 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.058+0000 7fc3a9e00640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac1f1c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c530 tx=0x7fc39c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:24.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.058+0000 7fc3a2000640 1 -- 172.21.15.105:0/1007700656 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c005ce0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.058+0000 7fc3a2000640 1 -- 172.21.15.105:0/1007700656 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c005e40 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.058+0000 7fc3a2000640 1 -- 172.21.15.105:0/1007700656 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0072b0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.058+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ac1f5b10 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.058+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ac1f60d0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.059+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ac1f62f0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.060+0000 7fc3a2000640 1 -- 172.21.15.105:0/1007700656 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc39c007410 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.061+0000 7fc3a2000640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc38c06ac00 0x7fc38c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.061+0000 7fc3a2000640 1 -- 172.21.15.105:0/1007700656 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc39c088cc0 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.061+0000 7fc3a9400640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc38c06ac00 0x7fc38c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.062+0000 7fc3a9400640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc38c06ac00 0x7fc38c06d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:24.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.066+0000 7fc3a2000640 1 -- 172.21.15.105:0/1007700656 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc39c08f080 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.299+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3ac04e030 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.300+0000 7fc3a2000640 1 -- 172.21.15.105:0/1007700656 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4739 (secure 0 0 0) 0x7fc39c017020 con 0x7fc3ac06a3a0 2024-03-23T07:56:24.300 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:24.300 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:22.457045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T07:56:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc38c06ac00 msgr2=0x7fc38c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc38c06ac00 0x7fc38c06d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398006bc0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 msgr2=0x7fc3ac1f1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac1f1c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c530 tx=0x7fc39c00c9f0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 shutdown_connections 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3ac0b08c0 0x7fc3ac1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc38c06ac00 0x7fc38c06d0c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3ac0ada70 0x7fc3ac0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 --2- 172.21.15.105:0/1007700656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3ac06a3a0 0x7fc3ac1f1c90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 >> 172.21.15.105:0/1007700656 conn(0x7fc3ac0b48e0 msgr2=0x7fc3ac158670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 shutdown_connections 2024-03-23T07:56:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.303+0000 7fc3ab200640 1 -- 172.21.15.105:0/1007700656 wait complete. 2024-03-23T07:56:24.322 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.527+0000 7f8694600640 1 Processor -- start 2024-03-23T07:56:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.528+0000 7f8694600640 1 -- start start 2024-03-23T07:56:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.529+0000 7f8694600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868c13faa0 0x7f868c160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.529+0000 7f8694600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f868c160800 0x7f868c0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.529+0000 7f8694600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c0b2030 con 0x7f868c0b1be0 2024-03-23T07:56:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.529+0000 7f8694600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0b38a0 con 0x7f868c160800 2024-03-23T07:56:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.529+0000 7f8694600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0b3b80 con 0x7f868c13faa0 2024-03-23T07:56:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.529+0000 7f8692600640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f868c0b1be0 0x7f868c1422b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58528/0 (socket says 172.21.15.105:58528) 2024-03-23T07:56:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.529+0000 7f8692600640 1 -- 172.21.15.105:0/896585762 learned_addr learned my addr 172.21.15.105:0/896585762 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f8691200640 1 --2- 172.21.15.105:0/896585762 >> v2:172.21.15.132:3300/0 conn(0x7f868c160800 0x7f868c0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f8691c00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868c13faa0 0x7f868c160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902333834 0 0) 0x7f8684001490 con 0x7f868c0b1be0 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8674003520 con 0x7f868c0b1be0 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421572414 0 0) 0x7f8674003520 con 0x7f868c0b1be0 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868c13faa0 msgr2=0x7f868c160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868c13faa0 0x7f868c160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 >> v2:172.21.15.132:3300/0 conn(0x7f868c160800 msgr2=0x7f868c0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 --2- 172.21.15.105:0/896585762 >> v2:172.21.15.132:3300/0 conn(0x7f868c160800 0x7f868c0b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c0b3e60 con 0x7f868c0b1be0 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f8684001b80 con 0x7f868c0b1be0 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 >> v1:172.21.15.105:6789/0 conn(0x7f868c0b1be0 legacy=0x7f868c1422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8674003970 0x7f8674003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.530+0000 7f868be00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868c13faa0 0x7f86740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f868be00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8674004d00 0x7f8674005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0b38a0 con 0x7f8674003970 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0b3b80 con 0x7f8674004d00 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867400b980 con 0x7f868c13faa0 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f8691c00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8674003970 0x7f8674003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f8691c00640 1 -- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868c13faa0 msgr2=0x7f86740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f8691c00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868c13faa0 0x7f86740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f8691c00640 1 -- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8674004d00 msgr2=0x7f8674005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f8691c00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8674004d00 0x7f8674005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.531+0000 7f8691c00640 1 -- 172.21.15.105:0/896585762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8674003520 con 0x7f8674003970 2024-03-23T07:56:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.532+0000 7f8691c00640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8674003970 0x7f8674003d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8680009ac0 tx=0x7f86740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca0dc36b7303c54 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868002c710 con 0x7f8674003970 2024-03-23T07:56:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868002c870 con 0x7f8674003970 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f868be00640 1 -- 172.21.15.105:0/896585762 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868002cc00 con 0x7f8674003970 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f8694600640 1 -- 172.21.15.105:0/896585762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8674003970 msgr2=0x7f8674003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f8694600640 1 --2- 172.21.15.105:0/896585762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8674003970 0x7f8674003d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8680009ac0 tx=0x7f86740057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f8692600640 1 -- 172.21.15.105:0/896585762 reap_dead start 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f8694600640 1 -- 172.21.15.105:0/896585762 shutdown_connections 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f8694600640 1 -- 172.21.15.105:0/896585762 >> 172.21.15.105:0/896585762 conn(0x7f868c06a3d0 msgr2=0x7f868c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f8694600640 1 -- 172.21.15.105:0/896585762 shutdown_connections 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.533+0000 7f8694600640 1 -- 172.21.15.105:0/896585762 wait complete. 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.535+0000 7f8694600640 1 Processor -- start 2024-03-23T07:56:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.535+0000 7f8694600640 1 -- start start 2024-03-23T07:56:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8694600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674003970 0x7f868c1f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8694600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f868c153b60 0x7f868c137b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8694600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c138c00 0x7f868c139080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8694600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1395c0 con 0x7f868c138c00 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8694600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c135740 con 0x7f868c153b60 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8694600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c138080 con 0x7f8674003970 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8692600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c138c00 0x7f868c139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8692600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c138c00 0x7f868c139080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53492/0 (socket says 172.21.15.105:53492) 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.536+0000 7f8692600640 1 -- 172.21.15.105:0/2657316774 learned_addr learned my addr 172.21.15.105:0/2657316774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8691c00640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674003970 0x7f868c1f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8691200640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f868c153b60 0x7f868c137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8692600640 1 -- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674003970 msgr2=0x7f868c1f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8692600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674003970 0x7f868c1f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8692600640 1 -- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f868c153b60 msgr2=0x7f868c137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8692600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f868c153b60 0x7f868c137b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8692600640 1 -- 172.21.15.105:0/2657316774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c13ad90 con 0x7f868c138c00 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8691200640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f868c153b60 0x7f868c137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8692600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c138c00 0x7f868c139080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8684000f80 tx=0x7f868400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8689600640 1 -- 172.21.15.105:0/2657316774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400a610 con 0x7f868c138c00 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8689600640 1 -- 172.21.15.105:0/2657316774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868400a790 con 0x7f868c138c00 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8689600640 1 -- 172.21.15.105:0/2657316774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400ab40 con 0x7f868c138c00 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c13b070 con 0x7f868c138c00 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.537+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c1f5d80 con 0x7f868c138c00 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.538+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c073390 con 0x7f868c138c00 2024-03-23T07:56:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.539+0000 7f8689600640 1 -- 172.21.15.105:0/2657316774 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f868400aca0 con 0x7f868c138c00 2024-03-23T07:56:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.541+0000 7f8689600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f867006ac00 0x7f867006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.541+0000 7f8689600640 1 -- 172.21.15.105:0/2657316774 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8684088d80 con 0x7f868c138c00 2024-03-23T07:56:24.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.541+0000 7f8691c00640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f867006ac00 0x7f867006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:24.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.542+0000 7f8691c00640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f867006ac00 0x7f867006d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8680036000 tx=0x7f8680030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.545+0000 7f8689600640 1 -- 172.21.15.105:0/2657316774 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8684053c70 con 0x7f868c138c00 2024-03-23T07:56:24.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.742+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f868c0b2030 con 0x7f868c138c00 2024-03-23T07:56:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.744+0000 7f8689600640 1 -- 172.21.15.105:0/2657316774 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7f868c0b2030 con 0x7f868c138c00 2024-03-23T07:56:24.744 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:24.744 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:24.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.746+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f867006ac00 msgr2=0x7f867006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.746+0000 7f8694600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f867006ac00 0x7f867006d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8680036000 tx=0x7f8680030000 comp rx=0 tx=0).stop 2024-03-23T07:56:24.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.746+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c138c00 msgr2=0x7f868c139080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.746+0000 7f8694600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c138c00 0x7f868c139080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8684000f80 tx=0x7f868400cf00 comp rx=0 tx=0).stop 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 shutdown_connections 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c138c00 0x7f868c139080 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f868c153b60 0x7f868c137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f867006ac00 0x7f867006d0c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 --2- 172.21.15.105:0/2657316774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674003970 0x7f868c1f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 >> 172.21.15.105:0/2657316774 conn(0x7f868c06a3d0 msgr2=0x7f868c15d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 shutdown_connections 2024-03-23T07:56:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:24.747+0000 7f8694600640 1 -- 172.21.15.105:0/2657316774 wait complete. 2024-03-23T07:56:25.168 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T07:56:17.694756+0000', 'modified': '2024-03-23T07:56:22.457045+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6834/713904573', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 713904573}, {'type': 'v1', 'addr': '172.21.15.132:6834', 'nonce': 713904573}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-23T07:56:25.168 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T07:56:25.168 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T07:56:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.383+0000 7fa1e5c00640 1 Processor -- start 2024-03-23T07:56:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.383+0000 7fa1e5c00640 1 -- start start 2024-03-23T07:56:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.384+0000 7fa1e5c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1e0159690 0x7fa1e00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.384+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 0x7fa1e00b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1e5c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e015e010 con 0x7fa1e00b1e10 2024-03-23T07:56:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1e5c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e00cc7f0 con 0x7fa1e0159690 2024-03-23T07:56:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1e5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00ccad0 con 0x7fa1e00cc330 2024-03-23T07:56:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1df400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa1e00b1e10 0x7fa1e00cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58532/0 (socket says 172.21.15.105:58532) 2024-03-23T07:56:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1df400640 1 -- 172.21.15.105:0/2948856464 learned_addr learned my addr 172.21.15.105:0/2948856464 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2948856464 >> v2:172.21.15.132:3300/0 conn(0x7fa1e0159690 0x7fa1e00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943058564 0 0) 0x7fa1e015e010 con 0x7fa1e00b1e10 2024-03-23T07:56:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1de000640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 0x7fa1e00b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.385+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c4003420 con 0x7fa1e00b1e10 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939081937 0 0) 0x7fa1c4003420 con 0x7fa1e00b1e10 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 msgr2=0x7fa1e00b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 0x7fa1e00b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 >> v2:172.21.15.132:3300/0 conn(0x7fa1e0159690 msgr2=0x7fa1e00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 --2- 172.21.15.105:0/2948856464 >> v2:172.21.15.132:3300/0 conn(0x7fa1e0159690 0x7fa1e00ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00ccdb0 con 0x7fa1e00b1e10 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fa1d80019f0 con 0x7fa1e00b1e10 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 >> v1:172.21.15.105:6789/0 conn(0x7fa1e00b1e10 legacy=0x7fa1e00cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 0x7fa1c4004a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1c4004fd0 0x7fa1c4003660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1c4003ba0 0x7fa1c4004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00cc7f0 con 0x7fa1c4003ba0 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00ccad0 con 0x7fa1c4004fd0 2024-03-23T07:56:25.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c400fde0 con 0x7fa1e00cc330 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1df400640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1c4003ba0 0x7fa1c4004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1de000640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1c4004fd0 0x7fa1c4003660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 0x7fa1c4004a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1df400640 1 -- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 msgr2=0x7fa1c4004a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1df400640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e00cc330 0x7fa1c4004a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1df400640 1 -- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1c4004fd0 msgr2=0x7fa1c4003660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1df400640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1c4004fd0 0x7fa1c4003660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.386+0000 7fa1df400640 1 -- 172.21.15.105:0/2948856464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4003420 con 0x7fa1c4003ba0 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.388+0000 7fa1df400640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1c4003ba0 0x7fa1c4004020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a860 tx=0x7fa1c40046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c90e8f2c931610 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.388+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8034630 con 0x7fa1c4003ba0 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.388+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d802f070 con 0x7fa1c4003ba0 2024-03-23T07:56:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.388+0000 7fa1dd600640 1 -- 172.21.15.105:0/2948856464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8034a30 con 0x7fa1c4003ba0 2024-03-23T07:56:25.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.388+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2948856464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1c4003ba0 msgr2=0x7fa1c4004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.388+0000 7fa1e5c00640 1 --2- 172.21.15.105:0/2948856464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1c4003ba0 0x7fa1c4004020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a860 tx=0x7fa1c40046e0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.388+0000 7fa1df400640 1 -- 172.21.15.105:0/2948856464 reap_dead start 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.389+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2948856464 shutdown_connections 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.389+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2948856464 >> 172.21.15.105:0/2948856464 conn(0x7fa1e006a0b0 msgr2=0x7fa1e006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.389+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2948856464 shutdown_connections 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.389+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2948856464 wait complete. 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.390+0000 7fa1e5c00640 1 Processor -- start 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.390+0000 7fa1e5c00640 1 -- start start 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.391+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1e0140820 0x7fa1e0143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.391+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e0140c00 0x7fa1e00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.391+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e0143690 0x7fa1e00cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.391+0000 7fa1e5c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0144190 con 0x7fa1e0140820 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.391+0000 7fa1e5c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0143bd0 con 0x7fa1e0143690 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.391+0000 7fa1e5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0143eb0 con 0x7fa1e0140c00 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1e0140820 0x7fa1e0143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1e0140820 0x7fa1e0143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53514/0 (socket says 172.21.15.105:53514) 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 -- 172.21.15.105:0/2691970770 learned_addr learned my addr 172.21.15.105:0/2691970770 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1de000640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e0140c00 0x7fa1e00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1df400640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e0143690 0x7fa1e00cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 -- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e0140c00 msgr2=0x7fa1e00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e0140c00 0x7fa1e00ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 -- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e0143690 msgr2=0x7fa1e00cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e0143690 0x7fa1e00cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 -- 172.21.15.105:0/2691970770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e01412b0 con 0x7fa1e0140820 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.392+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1e0140820 0x7fa1e0143150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00108e0 tx=0x7fa1d0010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.393+0000 7fa1d6a00640 1 -- 172.21.15.105:0/2691970770 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000eb60 con 0x7fa1e0140820 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.393+0000 7fa1d6a00640 1 -- 172.21.15.105:0/2691970770 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d000ecc0 con 0x7fa1e0140820 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.393+0000 7fa1d6a00640 1 -- 172.21.15.105:0/2691970770 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d001c400 con 0x7fa1e0140820 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.393+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e0141590 con 0x7fa1e0140820 2024-03-23T07:56:25.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.393+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e0157540 con 0x7fa1e0140820 2024-03-23T07:56:25.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.395+0000 7fa1d6a00640 1 -- 172.21.15.105:0/2691970770 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa1d001c560 con 0x7fa1e0140820 2024-03-23T07:56:25.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.396+0000 7fa1d6a00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa1c006ac30 0x7fa1c006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.397+0000 7fa1d6a00640 1 -- 172.21.15.105:0/2691970770 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa1d0089e50 con 0x7fa1e0140820 2024-03-23T07:56:25.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.397+0000 7fa1de000640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa1c006ac30 0x7fa1c006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.397+0000 7fa1e87cb740 1 Processor -- start 2024-03-23T07:56:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.397+0000 7fa1de000640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa1c006ac30 0x7fa1c006d0f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0140fe0 tx=0x7fa1cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.397+0000 7fa1e87cb740 1 -- start start 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.398+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e38d00 0x555a90e0bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.398+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e3ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.398+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e0c1e0 0x555a90e3bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.398+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90de72c0 con 0x555a90e390e0 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.398+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90d7c8f0 con 0x555a90e38d00 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.398+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a90d7a080 con 0x555a90e0c1e0 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.398+0000 7fa1de000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e3ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e3ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53522/0 (socket says 172.21.15.105:53522) 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 -- 172.21.15.105:0/63545618 learned_addr learned my addr 172.21.15.105:0/63545618 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1df400640 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e0c1e0 0x555a90e3bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 -- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e0c1e0 msgr2=0x555a90e3bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1dea00640 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e38d00 0x555a90e0bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e0c1e0 0x555a90e3bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 -- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e38d00 msgr2=0x555a90e0bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e38d00 0x555a90e0bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 -- 172.21.15.105:0/63545618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a90d40a10 con 0x555a90e390e0 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1de000640 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e3ba90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0108f0 tx=0x7fa1cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a57750cfe5b4da server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1cb400640 1 -- 172.21.15.105:0/63545618 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc015e00 con 0x555a90e390e0 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1cb400640 1 -- 172.21.15.105:0/63545618 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc01a070 con 0x555a90e390e0 2024-03-23T07:56:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.399+0000 7fa1cb400640 1 -- 172.21.15.105:0/63545618 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc021680 con 0x555a90e390e0 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 -- 172.21.15.105:0/63545618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 msgr2=0x555a90e3ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e3ba90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0108f0 tx=0x7fa1cc013000 comp rx=0 tx=0).stop 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 -- 172.21.15.105:0/63545618 shutdown_connections 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e0c1e0 0x555a90e3bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e3ba90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/63545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e38d00 0x555a90e0bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 -- 172.21.15.105:0/63545618 >> 172.21.15.105:0/63545618 conn(0x555a90e0df10 msgr2=0x555a90e56a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 -- 172.21.15.105:0/63545618 shutdown_connections 2024-03-23T07:56:25.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.400+0000 7fa1e87cb740 1 -- 172.21.15.105:0/63545618 wait complete. 2024-03-23T07:56:25.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.401+0000 7fa1e87cb740 1 -- start start 2024-03-23T07:56:25.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.401+0000 7fa1e87cb740 1 Processor -- start 2024-03-23T07:56:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e38d00 0x555a90e56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e390e0 0x555a90e3e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e3eb20 0x555a90e500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90d75970 con 0x555a90e38d00 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90d170a0 con 0x555a90e390e0 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a90d16d40 con 0x555a90e3eb20 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1e87cb740 10 client.0 _ll_register_callbacks cb 0x555a90d6efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e38d00 0x555a90e56a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e38d00 0x555a90e56a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53536/0 (socket says 172.21.15.105:53536) 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.402+0000 7fa1dea00640 1 -- 172.21.15.105:0/2480140877 learned_addr learned my addr 172.21.15.105:0/2480140877 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1de000640 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e390e0 0x555a90e3e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1dea00640 1 -- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e3eb20 msgr2=0x555a90e500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e3eb20 0x555a90e500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1dea00640 1 -- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e390e0 msgr2=0x555a90e3e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e390e0 0x555a90e3e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1dea00640 1 -- 172.21.15.105:0/2480140877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc00e050 con 0x555a90e38d00 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1de000640 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e390e0 0x555a90e3e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1dea00640 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e38d00 0x555a90e56a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0058910 tx=0x7fa1d0058a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d2733bdfcf1682 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1bfe00640 1 -- 172.21.15.105:0/2480140877 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0057e70 con 0x555a90e38d00 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1bfe00640 1 -- 172.21.15.105:0/2480140877 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa1d0057fd0 con 0x555a90e38d00 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.403+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a90d9fb10 con 0x555a90e38d00 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1bfe00640 1 -- 172.21.15.105:0/2480140877 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1d00608c0 con 0x555a90e38d00 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 10 client.4374 fetch_fsmap learned FSMap version 6 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x555a90d73ff0 con 0x555a90e38d00 2024-03-23T07:56:25.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1bfe00640 1 -- 172.21.15.105:0/2480140877 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a90d9fb10 con 0x555a90e38d00 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1bfe00640 1 -- 172.21.15.105:0/2480140877 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fa1d0024070 con 0x555a90e38d00 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 10 client.4374 fetch_fsmap finished waiting for FSMap version 6 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 10 client.4374 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 20 client.4374 populate_metadata read hostname 'smithi105' 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e29880 0x555a90e27a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 4 client.4374 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.404+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 --> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a90a693c0 con 0x555a90e29880 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.405+0000 7fa1de000640 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e29880 0x555a90e27a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.405+0000 7fa1de000640 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e29880 0x555a90e27a70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2eb658cad1988c server_cookie=52d56d0ba87fc363 in_seq=0 out_seq=0 2024-03-23T07:56:25.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.405+0000 7fa1bfe00640 10 client.4374 ms_handle_connect on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:25.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.407+0000 7fa1bfe00640 1 -- 172.21.15.105:0/2480140877 <== mds.0 v2:172.21.15.132:6832/713904573 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555a90a693c0 con 0x555a90e29880 2024-03-23T07:56:25.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.407+0000 7fa1bfe00640 10 client.4374 handle_command_reply: tid=0 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.407+0000 7fa1e87cb740 1 client.4374 shutdown 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e38d00 msgr2=0x555a90e56a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e38d00 0x555a90e56a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0058910 tx=0x7fa1d0058a40 comp rx=0 tx=0).stop 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 shutdown_connections 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e3eb20 0x555a90e500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e390e0 0x555a90e3e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e29880 0x555a90e27a70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/2480140877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e38d00 0x555a90e56a70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 >> 172.21.15.105:0/2480140877 conn(0x555a90e0c1e0 msgr2=0x555a90e59fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1bfe00640 0 client.4374 ms_handle_reset on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 shutdown_connections 2024-03-23T07:56:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 1 -- 172.21.15.105:0/2480140877 wait complete. 2024-03-23T07:56:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.408+0000 7fa1e87cb740 20 client.4374 trim_cache size 0 max 16384 2024-03-23T07:56:25.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.424+0000 7fa1e87cb740 1 Processor -- start 2024-03-23T07:56:25.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.424+0000 7fa1e87cb740 1 -- start start 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 0x555a90e0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e4f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90e4ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90adcf20 con 0x555a90e390e0 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90e50ab0 con 0x555a90e3eb20 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a90e26dd0 con 0x555a90e38d00 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e4f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e4f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53542/0 (socket says 172.21.15.105:53542) 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 -- 172.21.15.105:0/4077221159 learned_addr learned my addr 172.21.15.105:0/4077221159 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 -- 172.21.15.105:0/4077221159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 msgr2=0x555a90e0b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 --2- 172.21.15.105:0/4077221159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 0x555a90e0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 -- 172.21.15.105:0/4077221159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 msgr2=0x555a90e4ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 --2- 172.21.15.105:0/4077221159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90e4ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.425+0000 7fa1de000640 1 -- 172.21.15.105:0/4077221159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a90da97c0 con 0x555a90e390e0 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.426+0000 7fa1de000640 1 --2- 172.21.15.105:0/4077221159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e4f9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0108c0 tx=0x7fa1cc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f406a809f204501 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.426+0000 7fa1cbe00640 1 -- 172.21.15.105:0/4077221159 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc032040 con 0x555a90e390e0 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.426+0000 7fa1cbe00640 1 -- 172.21.15.105:0/4077221159 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc015d00 con 0x555a90e390e0 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.426+0000 7fa1cbe00640 1 -- 172.21.15.105:0/4077221159 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc034690 con 0x555a90e390e0 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 -- 172.21.15.105:0/4077221159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 msgr2=0x555a90e4f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/4077221159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e4f9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0108c0 tx=0x7fa1cc015000 comp rx=0 tx=0).stop 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 -- 172.21.15.105:0/4077221159 shutdown_connections 2024-03-23T07:56:25.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/4077221159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90e4ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/4077221159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e390e0 0x555a90e4f9f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/4077221159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 0x555a90e0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 -- 172.21.15.105:0/4077221159 >> 172.21.15.105:0/4077221159 conn(0x555a90e0c1e0 msgr2=0x555a90e073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 -- 172.21.15.105:0/4077221159 shutdown_connections 2024-03-23T07:56:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 -- 172.21.15.105:0/4077221159 wait complete. 2024-03-23T07:56:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.427+0000 7fa1e87cb740 1 -- start start 2024-03-23T07:56:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.428+0000 7fa1e87cb740 1 Processor -- start 2024-03-23T07:56:25.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.428+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 0x555a90ebffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90ebf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1e87cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e78ac0 0x555a90e0ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90ebfde0 con 0x555a90e78ac0 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a90ec0580 con 0x555a90e3eb20 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1e87cb740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a90e6fa90 con 0x555a90e38d00 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1e87cb740 10 client.0 _ll_register_callbacks cb 0x7fa1d0004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e78ac0 0x555a90e0ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e78ac0 0x555a90e0ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53544/0 (socket says 172.21.15.105:53544) 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 -- 172.21.15.105:0/466786811 learned_addr learned my addr 172.21.15.105:0/466786811 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1de000640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90ebf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1dea00640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 0x555a90ebffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 -- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 msgr2=0x555a90ebffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 0x555a90ebffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 -- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 msgr2=0x555a90ebf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90ebf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 -- 172.21.15.105:0/466786811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a90d40a10 con 0x555a90e78ac0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1de000640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90ebf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.429+0000 7fa1df400640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e78ac0 0x555a90e0ace0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a950 tx=0x7fa1d800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd5e05f9da570f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1bfe00640 1 -- 172.21.15.105:0/466786811 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80019c0 con 0x555a90e78ac0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1bfe00640 1 -- 172.21.15.105:0/466786811 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa1d8034430 con 0x555a90e78ac0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1cc01a070 con 0x555a90e78ac0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1bfe00640 1 -- 172.21.15.105:0/466786811 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1d8001fd0 con 0x555a90e78ac0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 10 client.4380 fetch_fsmap learned FSMap version 6 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x555a90d162b0 con 0x555a90e78ac0 2024-03-23T07:56:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1bfe00640 1 -- 172.21.15.105:0/466786811 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc01a070 con 0x555a90e78ac0 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1bfe00640 1 -- 172.21.15.105:0/466786811 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fa1d8047350 con 0x555a90e78ac0 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 10 client.4380 fetch_fsmap finished waiting for FSMap version 6 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 10 client.4380 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 20 client.4380 populate_metadata read hostname 'smithi105' 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e4cdb0 0x555a90e685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 4 client.4380 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.430+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 --> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a90a7a6c0 con 0x555a90e4cdb0 2024-03-23T07:56:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.431+0000 7fa1dea00640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e4cdb0 0x555a90e685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.432+0000 7fa1dea00640 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e4cdb0 0x555a90e685d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bf8e26bb1cf7809 server_cookie=9f6bf13451a405bf in_seq=0 out_seq=0 2024-03-23T07:56:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.432+0000 7fa1bfe00640 10 client.4380 ms_handle_connect on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.432+0000 7fa1bfe00640 1 -- 172.21.15.105:0/466786811 <== mds.0 v2:172.21.15.132:6832/713904573 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x555a90a7a6c0 con 0x555a90e4cdb0 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.432+0000 7fa1bfe00640 10 client.4380 handle_command_reply: tid=0 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.432+0000 7fa1e87cb740 1 client.4380 shutdown 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e78ac0 msgr2=0x555a90e0ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e78ac0 0x555a90e0ace0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a950 tx=0x7fa1d800a660 comp rx=0 tx=0).stop 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 shutdown_connections 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x555a90e4cdb0 0x555a90e685d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a90e78ac0 0x555a90e0ace0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555a90e3eb20 0x555a90ebf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 --2- 172.21.15.105:0/466786811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555a90e38d00 0x555a90ebffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1bfe00640 0 client.4380 ms_handle_reset on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 >> 172.21.15.105:0/466786811 conn(0x555a90e0c1e0 msgr2=0x555a90e4f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 shutdown_connections 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 1 -- 172.21.15.105:0/466786811 wait complete. 2024-03-23T07:56:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.433+0000 7fa1e87cb740 20 client.4380 trim_cache size 0 max 16384 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "e641b836-a0e5-4e95-93aa-bb0628868335", 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4282, 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 7.7307781909999997, 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 6, 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 26, 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 25, 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 12.389571569999999 2024-03-23T07:56:25.436 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:56:25.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa1c006ac30 msgr2=0x7fa1c006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa1c006ac30 0x7fa1c006d0f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0140fe0 tx=0x7fa1cc008040 comp rx=0 tx=0).stop 2024-03-23T07:56:25.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1e0140820 msgr2=0x7fa1e0143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1e0140820 0x7fa1e0143150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00108e0 tx=0x7fa1d0010da0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 shutdown_connections 2024-03-23T07:56:25.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e0143690 0x7fa1e00cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa1c006ac30 0x7fa1c006d0f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1e0140c00 0x7fa1e00ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 --2- 172.21.15.105:0/2691970770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1e0140820 0x7fa1e0143150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 >> 172.21.15.105:0/2691970770 conn(0x7fa1e006a0b0 msgr2=0x7fa1e00b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 shutdown_connections 2024-03-23T07:56:25.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.434+0000 7fa1e5c00640 1 -- 172.21.15.105:0/2691970770 wait complete. 2024-03-23T07:56:25.451 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T07:56:25.451 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:25.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T07:56:25.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.647+0000 7f98e9200640 1 Processor -- start 2024-03-23T07:56:25.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.647+0000 7f98e9200640 1 -- start start 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e9200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98e40b5a30 0x7f98e40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e9200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98e40b4350 0x7f98e40b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e9200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e40d1830 con 0x7f98e415cda0 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e9200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e40b4d00 con 0x7f98e40b5a30 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e9200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e40b1f20 con 0x7f98e40b4350 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e1e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f98e415cda0 0x7f98e40cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58540/0 (socket says 172.21.15.105:58540) 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e1e00640 1 -- 172.21.15.105:0/3877612067 learned_addr learned my addr 172.21.15.105:0/3877612067 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e1400640 1 --2- 172.21.15.105:0/3877612067 >> v2:172.21.15.105:3300/0 conn(0x7f98e40b5a30 0x7f98e40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e2800640 1 --2- 172.21.15.105:0/3877612067 >> v2:172.21.15.132:3300/0 conn(0x7f98e40b4350 0x7f98e40b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e1400640 1 -- 172.21.15.105:0/3877612067 >> v2:172.21.15.132:3300/0 conn(0x7f98e40b4350 msgr2=0x7f98e40b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:25.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e1400640 1 --2- 172.21.15.105:0/3877612067 >> v2:172.21.15.132:3300/0 conn(0x7f98e40b4350 0x7f98e40b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e1400640 1 -- 172.21.15.105:0/3877612067 >> v1:172.21.15.105:6789/0 conn(0x7f98e415cda0 legacy=0x7f98e40cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.650+0000 7f98e1400640 1 -- 172.21.15.105:0/3877612067 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40d0c20 con 0x7f98e40b5a30 2024-03-23T07:56:25.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.649+0000 7f98e0a00640 1 -- 172.21.15.105:0/3877612067 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155510719 0 0) 0x7f98e40d1830 con 0x7f98e415cda0 2024-03-23T07:56:25.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.651+0000 7f98e1400640 1 --2- 172.21.15.105:0/3877612067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e40b5a30 0x7f98e40b5e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98e40b8460 tx=0x7f98cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbae291a7f63d293 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.651+0000 7f98e0a00640 1 -- 172.21.15.105:0/3877612067 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc031830 con 0x7f98e40b5a30 2024-03-23T07:56:25.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.651+0000 7f98e0a00640 1 -- 172.21.15.105:0/3877612067 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc031990 con 0x7f98e40b5a30 2024-03-23T07:56:25.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.651+0000 7f98e0a00640 1 -- 172.21.15.105:0/3877612067 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc031d20 con 0x7f98e40b5a30 2024-03-23T07:56:25.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 -- 172.21.15.105:0/3877612067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e40b5a30 msgr2=0x7f98e40b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 --2- 172.21.15.105:0/3877612067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e40b5a30 0x7f98e40b5e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98e40b8460 tx=0x7f98cc02b8d0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 -- 172.21.15.105:0/3877612067 shutdown_connections 2024-03-23T07:56:25.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 --2- 172.21.15.105:0/3877612067 >> v2:172.21.15.132:3300/0 conn(0x7f98e40b4350 0x7f98e40b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 --2- 172.21.15.105:0/3877612067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e40b5a30 0x7f98e40b5e90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 -- 172.21.15.105:0/3877612067 >> 172.21.15.105:0/3877612067 conn(0x7f98e40cf350 msgr2=0x7f98e40cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 -- 172.21.15.105:0/3877612067 shutdown_connections 2024-03-23T07:56:25.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.652+0000 7f98e9200640 1 -- 172.21.15.105:0/3877612067 wait complete. 2024-03-23T07:56:25.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.653+0000 7f98e9200640 1 Processor -- start 2024-03-23T07:56:25.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.654+0000 7f98e9200640 1 -- start start 2024-03-23T07:56:25.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.654+0000 7f98e9200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b4350 0x7f98e41502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.654+0000 7f98e9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b5a30 0x7f98e41f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e415cda0 0x7f98e40d2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e9200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40c7b90 con 0x7f98e415cda0 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e9200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40c7de0 con 0x7f98e40b4350 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e9200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40c80c0 con 0x7f98e40b5a30 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e415cda0 0x7f98e40d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e415cda0 0x7f98e40d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53564/0 (socket says 172.21.15.105:53564) 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 -- 172.21.15.105:0/4222098463 learned_addr learned my addr 172.21.15.105:0/4222098463 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e1e00640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b4350 0x7f98e41502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e1400640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b5a30 0x7f98e41f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 -- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b5a30 msgr2=0x7f98e41f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b5a30 0x7f98e41f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 -- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b4350 msgr2=0x7f98e41502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b4350 0x7f98e41502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 -- 172.21.15.105:0/4222098463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009400 con 0x7f98e415cda0 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e1e00640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b4350 0x7f98e41502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.655+0000 7f98e2800640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e415cda0 0x7f98e40d2310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007e40 tx=0x7f98d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.656+0000 7f98da000640 1 -- 172.21.15.105:0/4222098463 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4014620 con 0x7f98e415cda0 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.656+0000 7f98da000640 1 -- 172.21.15.105:0/4222098463 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400ee90 con 0x7f98e415cda0 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.656+0000 7f98da000640 1 -- 172.21.15.105:0/4222098463 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40149e0 con 0x7f98e415cda0 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.656+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e415d220 con 0x7f98e415cda0 2024-03-23T07:56:25.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.656+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e4150fe0 con 0x7f98e415cda0 2024-03-23T07:56:25.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.657+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e415d740 con 0x7f98e415cda0 2024-03-23T07:56:25.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.658+0000 7f98da000640 1 -- 172.21.15.105:0/4222098463 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f98d4014b40 con 0x7f98e415cda0 2024-03-23T07:56:25.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.659+0000 7f98da000640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f98c006ac30 0x7f98c006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:25.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.659+0000 7f98da000640 1 -- 172.21.15.105:0/4222098463 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f98d4089fe0 con 0x7f98e415cda0 2024-03-23T07:56:25.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.659+0000 7f98e1e00640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f98c006ac30 0x7f98c006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:25.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.660+0000 7f98e1e00640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f98c006ac30 0x7f98c006d0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000f60 tx=0x7f98dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:25.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.664+0000 7f98da000640 1 -- 172.21.15.105:0/4222098463 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f98d4054ec0 con 0x7f98e415cda0 2024-03-23T07:56:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.902+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98e4066500 con 0x7f98e415cda0 2024-03-23T07:56:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.903+0000 7f98da000640 1 -- 172.21.15.105:0/4222098463 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f98d40584e0 con 0x7f98e415cda0 2024-03-23T07:56:25.902 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-23T07:56:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f98c006ac30 msgr2=0x7f98c006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f98c006ac30 0x7f98c006d0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000f60 tx=0x7f98dc009040 comp rx=0 tx=0).stop 2024-03-23T07:56:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e415cda0 msgr2=0x7f98e40d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e415cda0 0x7f98e40d2310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007e40 tx=0x7f98d400e4b0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 shutdown_connections 2024-03-23T07:56:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98e415cda0 0x7f98e40d2310 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f98c006ac30 0x7f98c006d0f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b5a30 0x7f98e41f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 --2- 172.21.15.105:0/4222098463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b4350 0x7f98e41502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.905+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 >> 172.21.15.105:0/4222098463 conn(0x7f98e40cf350 msgr2=0x7f98e40cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.906+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 shutdown_connections 2024-03-23T07:56:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:25.906+0000 7f98e9200640 1 -- 172.21.15.105:0/4222098463 wait complete. 2024-03-23T07:56:25.923 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-23T07:56:25.923 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T07:56:25.923 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-23T07:56:25.924 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-23T07:56:25.924 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:25.924 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-23T07:56:25.924 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T07:56:25.924 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T07:56:25.924 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-23T07:56:25.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:25.924 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:56:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42539sec preferred_lft 42539sec 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-23T07:56:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:56:25.975 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T07:56:25.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:25.975 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:56:25.975 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:56:25.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-03-23T07:56:25.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-03-23T07:56:25.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-03-23T07:56:25.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T07:56:25.975 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:56:26.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:26.416 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T07:56:26.496 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-23T07:56:26.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:26.499 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-23T07:56:26.553 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-23T07:56:26.553 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T07:56:26.554 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T07:56:26.554 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T07:56:26.554 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T07:56:26.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:26.554 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:56:26.554 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:56:26.554 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T07:56:26.555 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T07:56:26.555 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T07:56:26.555 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:56:27.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:27.234 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:56:27.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:27.288 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-03-23T07:56:27.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:27.342 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:56:27.342 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:56:27.342 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-23T07:56:27.342 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-23T07:56:27.342 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:56:27.479 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-23T07:56:27.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:27.479 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:56:27.479 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:56:27.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-23T07:56:27.480 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T07:56:27.480 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-23T07:56:27.480 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-23T07:56:27.480 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-23T07:56:27.480 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:56:27.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:27.764 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:56:27.764 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:56:27.764 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-03-23T07:56:27.764 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T07:56:27.764 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:56:27.923 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-23T07:56:27.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:56:27.924 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:27.970 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-23T07:56:27.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:56:27.970 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:28.065 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-03-23T07:56:28.157 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T07:56:28.158 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T07:56:28.245 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T07:56:28.248 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T07:56:28.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:28.249 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-03-23T07:56:28.293 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T07:56:28.294 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-23T07:56:28.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-03-23T07:56:28.418 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T07:56:28.418 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T07:56:28.485 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T07:56:28.488 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T07:56:28.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:28.489 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-03-23T07:56:28.526 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:2024-03-23T07:56:28.527+0000 7fe7a3a3a480 -1 init, newargv = 0x55eb7b892ab0 newargc=15 2024-03-23T07:56:28.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[43624]: starting ceph client 2024-03-23T07:56:28.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[43624]: starting fuse 2024-03-23T07:56:29.628 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-03-23T07:56:29.729 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T07:56:29.730 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T07:56:29.816 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T07:56:29.820 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T07:56:29.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:29.821 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-03-23T07:56:29.872 INFO:teuthology.orchestra.run.smithi105.stdout:41 2024-03-23T07:56:29.872 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-23T07:56:29.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T07:56:29.872 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-23T07:56:29.873 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.admin")) 2024-03-23T07:56:29.874 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-03-23T07:56:30.042 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.admin.43624.asok 2024-03-23T07:56:30.219 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43624.asok 2024-03-23T07:56:30.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:30.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43624.asok status 2024-03-23T07:56:30.456 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:56:30.456 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "82316b04821a6c2c1d458a881920454bbadb8837", 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "admin", 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "43624", 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4395, 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4395 2024-03-23T07:56:30.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1902833330 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1902833330 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4395 192.168.0.1:0/1902833330", 2024-03-23T07:56:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/1902833330", 2024-03-23T07:56:30.459 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2024-03-23T07:56:30.459 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 6, 2024-03-23T07:56:30.459 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 26, 2024-03-23T07:56:30.459 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-03-23T07:56:30.459 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-03-23T07:56:30.459 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-03-23T07:56:30.459 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:56:30.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:30.471 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2024-03-23T07:56:30.471 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:30.524 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2024-03-23T07:56:30.524 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:30.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:30.524 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:30.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:30.618 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-23T07:56:30.704 INFO:teuthology.orchestra.run.smithi105.stdout:# file: . 2024-03-23T07:56:30.704 INFO:teuthology.orchestra.run.smithi105.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-23T07:56:30.704 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:30.706 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:56:30.706 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:56:30.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:30.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.972+0000 7fae28c00640 1 Processor -- start 2024-03-23T07:56:30.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.972+0000 7fae28c00640 1 -- start start 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.973+0000 7fae28c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae240b39b0 0x7fae240b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.973+0000 7fae28c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415c400 0x7fae2415c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.973+0000 7fae28c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2406c6f0 con 0x7fae2415b560 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.973+0000 7fae28c00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae240b4370 con 0x7fae240b39b0 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.973+0000 7fae28c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2406c9d0 con 0x7fae2415c400 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae1be00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae240b39b0 0x7fae240b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae1be00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae240b39b0 0x7fae240b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41288/0 (socket says 172.21.15.105:41288) 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae1be00640 1 -- 172.21.15.105:0/2321058675 learned_addr learned my addr 172.21.15.105:0/2321058675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:30.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae22800640 1 --2- 172.21.15.105:0/2321058675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415c400 0x7fae2415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae1be00640 1 -- 172.21.15.105:0/2321058675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415c400 msgr2=0x7fae2415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae1be00640 1 --2- 172.21.15.105:0/2321058675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415c400 0x7fae2415c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae1be00640 1 -- 172.21.15.105:0/2321058675 >> v1:172.21.15.132:6789/0 conn(0x7fae2415b560 legacy=0x7fae2415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae1be00640 1 -- 172.21.15.105:0/2321058675 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2406ccb0 con 0x7fae240b39b0 2024-03-23T07:56:30.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.974+0000 7fae22800640 1 --2- 172.21.15.105:0/2321058675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415c400 0x7fae2415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.976+0000 7fae1be00640 1 --2- 172.21.15.105:0/2321058675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240b3e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009870 tx=0x7fae0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d99c7f63fc8575d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.976+0000 7fae21400640 1 -- 172.21.15.105:0/2321058675 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c030840 con 0x7fae240b39b0 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.976+0000 7fae21400640 1 -- 172.21.15.105:0/2321058675 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c0309a0 con 0x7fae240b39b0 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.976+0000 7fae21400640 1 -- 172.21.15.105:0/2321058675 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c030d30 con 0x7fae240b39b0 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.976+0000 7fae28c00640 1 -- 172.21.15.105:0/2321058675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 msgr2=0x7fae240b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.976+0000 7fae28c00640 1 --2- 172.21.15.105:0/2321058675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240b3e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009870 tx=0x7fae0c02b980 comp rx=0 tx=0).stop 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.977+0000 7fae28c00640 1 -- 172.21.15.105:0/2321058675 shutdown_connections 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.977+0000 7fae28c00640 1 --2- 172.21.15.105:0/2321058675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415c400 0x7fae2415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:30.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.977+0000 7fae28c00640 1 --2- 172.21.15.105:0/2321058675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240b3e30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:30.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.977+0000 7fae28c00640 1 -- 172.21.15.105:0/2321058675 >> 172.21.15.105:0/2321058675 conn(0x7fae2406a3d0 msgr2=0x7fae240b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:30.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.977+0000 7fae28c00640 1 -- 172.21.15.105:0/2321058675 shutdown_connections 2024-03-23T07:56:30.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.977+0000 7fae28c00640 1 -- 172.21.15.105:0/2321058675 wait complete. 2024-03-23T07:56:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.978+0000 7fae28c00640 1 Processor -- start 2024-03-23T07:56:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.978+0000 7fae28c00640 1 -- start start 2024-03-23T07:56:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.979+0000 7fae28c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.979+0000 7fae28c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415b560 0x7fae2414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.979+0000 7fae28c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae2415c400 0x7fae24155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.979+0000 7fae28c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae241f00f0 con 0x7fae240b39b0 2024-03-23T07:56:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.979+0000 7fae28c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae241f03d0 con 0x7fae2415c400 2024-03-23T07:56:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.979+0000 7fae28c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae241f06b0 con 0x7fae2415b560 2024-03-23T07:56:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41292/0 (socket says 172.21.15.105:41292) 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 -- 172.21.15.105:0/860923391 learned_addr learned my addr 172.21.15.105:0/860923391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae1be00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415b560 0x7fae2414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 -- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415b560 msgr2=0x7fae2414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae22800640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae2415c400 0x7fae24155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415b560 0x7fae2414b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 -- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae2415c400 msgr2=0x7fae24155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae2415c400 0x7fae24155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 -- 172.21.15.105:0/860923391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0094d0 con 0x7fae240b39b0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.980+0000 7fae21e00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240cc590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c450 tx=0x7fae1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.981+0000 7fae1a000640 1 -- 172.21.15.105:0/860923391 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140052d0 con 0x7fae240b39b0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.981+0000 7fae1a000640 1 -- 172.21.15.105:0/860923391 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14005430 con 0x7fae240b39b0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.981+0000 7fae1a000640 1 -- 172.21.15.105:0/860923391 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14012870 con 0x7fae240b39b0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.981+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae241f0990 con 0x7fae240b39b0 2024-03-23T07:56:30.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.981+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae240ccad0 con 0x7fae240b39b0 2024-03-23T07:56:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.982+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2406ac20 con 0x7fae240b39b0 2024-03-23T07:56:30.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.983+0000 7fae1a000640 1 -- 172.21.15.105:0/860923391 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fae140129d0 con 0x7fae240b39b0 2024-03-23T07:56:30.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.984+0000 7fae1a000640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fae0006ac00 0x7fae0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:30.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.984+0000 7fae1a000640 1 -- 172.21.15.105:0/860923391 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fae14089250 con 0x7fae240b39b0 2024-03-23T07:56:30.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.984+0000 7fae1be00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fae0006ac00 0x7fae0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.985+0000 7fae1be00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fae0006ac00 0x7fae0006d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae0c036000 tx=0x7fae0c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:30.989+0000 7fae1a000640 1 -- 172.21.15.105:0/860923391 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fae140540a0 con 0x7fae240b39b0 2024-03-23T07:56:31.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.184+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae241f0230 con 0x7fae240b39b0 2024-03-23T07:56:31.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.186+0000 7fae1a000640 1 -- 172.21.15.105:0/860923391 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7fae241f0230 con 0x7fae240b39b0 2024-03-23T07:56:31.186 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:31.186 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:31.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fae0006ac00 msgr2=0x7fae0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fae0006ac00 0x7fae0006d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae0c036000 tx=0x7fae0c03b040 comp rx=0 tx=0).stop 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 msgr2=0x7fae240cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240cc590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c450 tx=0x7fae1400c910 comp rx=0 tx=0).stop 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 shutdown_connections 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae2415c400 0x7fae24155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fae0006ac00 0x7fae0006d0c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae2415b560 0x7fae2414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 --2- 172.21.15.105:0/860923391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae240b39b0 0x7fae240cc590 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 >> 172.21.15.105:0/860923391 conn(0x7fae2406a3d0 msgr2=0x7fae24158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 shutdown_connections 2024-03-23T07:56:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.189+0000 7fae28c00640 1 -- 172.21.15.105:0/860923391 wait complete. 2024-03-23T07:56:31.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:56:31.209 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:56:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.421+0000 7f8172000640 1 Processor -- start 2024-03-23T07:56:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.421+0000 7f8172000640 1 -- start start 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.422+0000 7f8172000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f816c159830 0x7f816c0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.422+0000 7f8172000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ca7b0 0x7f816c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f8172000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c159c10 con 0x7f816c0ce930 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f8172000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816c06de50 con 0x7f816c159830 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f8172000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c0cd840 con 0x7f816c0ca7b0 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816b400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f816c159830 0x7f816c0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816be00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f816c0ce930 0x7f816c0b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43636/0 (socket says 172.21.15.105:43636) 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816be00640 1 -- 172.21.15.105:0/1946727148 learned_addr learned my addr 172.21.15.105:0/1946727148 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816aa00640 1 --2- 172.21.15.105:0/1946727148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ca7b0 0x7f816c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816a000640 1 -- 172.21.15.105:0/1946727148 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970322377 0 0) 0x7f816c159c10 con 0x7f816c0ce930 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816b400640 1 -- 172.21.15.105:0/1946727148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ca7b0 msgr2=0x7f816c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816b400640 1 --2- 172.21.15.105:0/1946727148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ca7b0 0x7f816c0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816b400640 1 -- 172.21.15.105:0/1946727148 >> v1:172.21.15.105:6789/0 conn(0x7f816c0ce930 legacy=0x7f816c0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.423+0000 7f816b400640 1 -- 172.21.15.105:0/1946727148 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0ccc30 con 0x7f816c159830 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.424+0000 7f816b400640 1 --2- 172.21.15.105:0/1946727148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c159830 0x7f816c0cc3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f815c0097a0 tx=0x7f815c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f001f3e5e062facf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f816a000640 1 -- 172.21.15.105:0/1946727148 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c030610 con 0x7f816c159830 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f816a000640 1 -- 172.21.15.105:0/1946727148 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c030770 con 0x7f816c159830 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f816a000640 1 -- 172.21.15.105:0/1946727148 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c030b00 con 0x7f816c159830 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 -- 172.21.15.105:0/1946727148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c159830 msgr2=0x7f816c0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 --2- 172.21.15.105:0/1946727148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c159830 0x7f816c0cc3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f815c0097a0 tx=0x7f815c02b960 comp rx=0 tx=0).stop 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 -- 172.21.15.105:0/1946727148 shutdown_connections 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 --2- 172.21.15.105:0/1946727148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ca7b0 0x7f816c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 --2- 172.21.15.105:0/1946727148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c159830 0x7f816c0cc3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 -- 172.21.15.105:0/1946727148 >> 172.21.15.105:0/1946727148 conn(0x7f816c06a0b0 msgr2=0x7f816c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 -- 172.21.15.105:0/1946727148 shutdown_connections 2024-03-23T07:56:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.425+0000 7f8172000640 1 -- 172.21.15.105:0/1946727148 wait complete. 2024-03-23T07:56:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.427+0000 7f8172000640 1 Processor -- start 2024-03-23T07:56:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.427+0000 7f8172000640 1 -- start start 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f8172000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c0ca7b0 0x7f816c146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f8172000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ce930 0x7f816c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f8172000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816c159830 0x7f816c0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f8172000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c144fd0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f8172000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c15c5c0 con 0x7f816c159830 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f8172000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c15c8a0 con 0x7f816c0ce930 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f816b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c0ca7b0 0x7f816c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f816b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c0ca7b0 0x7f816c146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41300/0 (socket says 172.21.15.105:41300) 2024-03-23T07:56:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.428+0000 7f816b400640 1 -- 172.21.15.105:0/2961696688 learned_addr learned my addr 172.21.15.105:0/2961696688 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f816b400640 1 -- 172.21.15.105:0/2961696688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ce930 msgr2=0x7f816c15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f816b400640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ce930 0x7f816c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f816b400640 1 -- 172.21.15.105:0/2961696688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816c159830 msgr2=0x7f816c0ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f816b400640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816c159830 0x7f816c0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f816b400640 1 -- 172.21.15.105:0/2961696688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c009400 con 0x7f816c0ca7b0 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f816b400640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c0ca7b0 0x7f816c146510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f815c0098c0 tx=0x7f815c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f8163400640 1 -- 172.21.15.105:0/2961696688 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c03e070 con 0x7f816c0ca7b0 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f8163400640 1 -- 172.21.15.105:0/2961696688 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c039800 con 0x7f816c0ca7b0 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f8163400640 1 -- 172.21.15.105:0/2961696688 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c039bb0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.429+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c15cb80 con 0x7f816c0ca7b0 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.430+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c0ae8b0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.431+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816c1454b0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.432+0000 7f8163400640 1 -- 172.21.15.105:0/2961696688 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f815c042070 con 0x7f816c0ca7b0 2024-03-23T07:56:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.433+0000 7f8163400640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f814c06ac30 0x7f814c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.433+0000 7f8163400640 1 -- 172.21.15.105:0/2961696688 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f815c0ab2a0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.433+0000 7f816aa00640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f814c06ac30 0x7f814c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.434+0000 7f816aa00640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f814c06ac30 0x7f814c06d0f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8158006fd0 tx=0x7f8158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.437+0000 7f8163400640 1 -- 172.21.15.105:0/2961696688 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f816c1454b0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.630+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f816c1454b0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.631+0000 7f8163400640 1 -- 172.21.15.105:0/2961696688 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f816c1454b0 con 0x7f816c0ca7b0 2024-03-23T07:56:31.630 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:56:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f814c06ac30 msgr2=0x7f814c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f814c06ac30 0x7f814c06d0f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8158006fd0 tx=0x7f8158008040 comp rx=0 tx=0).stop 2024-03-23T07:56:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c0ca7b0 msgr2=0x7f816c146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c0ca7b0 0x7f816c146510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f815c0098c0 tx=0x7f815c0046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 shutdown_connections 2024-03-23T07:56:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816c159830 0x7f816c0ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f814c06ac30 0x7f814c06d0f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816c0ce930 0x7f816c15bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 --2- 172.21.15.105:0/2961696688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c0ca7b0 0x7f816c146510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.633+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 >> 172.21.15.105:0/2961696688 conn(0x7f816c06a0b0 msgr2=0x7f816c0b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.634+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 shutdown_connections 2024-03-23T07:56:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.634+0000 7f8172000640 1 -- 172.21.15.105:0/2961696688 wait complete. 2024-03-23T07:56:31.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:56:31.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.865+0000 7ff38d200640 1 Processor -- start 2024-03-23T07:56:31.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.865+0000 7ff38d200640 1 -- start start 2024-03-23T07:56:31.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.867+0000 7ff38d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3880ce930 0x7ff3880add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.867+0000 7ff38d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff38806ce90 0x7ff38806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.867+0000 7ff38d200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3880cdba0 con 0x7ff3880cd3e0 2024-03-23T07:56:31.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.867+0000 7ff38d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38806c760 con 0x7ff38806ce90 2024-03-23T07:56:31.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.867+0000 7ff38d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880b42d0 con 0x7ff3880ce930 2024-03-23T07:56:31.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.867+0000 7ff385e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3880ce930 0x7ff3880add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.867+0000 7ff386800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff38806ce90 0x7ff38806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.868+0000 7ff385e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3880ce930 0x7ff3880add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:60394/0 (socket says 172.21.15.105:60394) 2024-03-23T07:56:31.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.868+0000 7ff385e00640 1 -- 172.21.15.105:0/4226784792 learned_addr learned my addr 172.21.15.105:0/4226784792 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:31.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.868+0000 7ff385e00640 1 -- 172.21.15.105:0/4226784792 >> v2:172.21.15.132:3300/0 conn(0x7ff38806ce90 msgr2=0x7ff38806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:31.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.868+0000 7ff385e00640 1 --2- 172.21.15.105:0/4226784792 >> v2:172.21.15.132:3300/0 conn(0x7ff38806ce90 0x7ff38806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.868+0000 7ff385e00640 1 -- 172.21.15.105:0/4226784792 >> v1:172.21.15.132:6789/0 conn(0x7ff3880cd3e0 legacy=0x7ff3880ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.868+0000 7ff385e00640 1 -- 172.21.15.105:0/4226784792 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3880cde80 con 0x7ff3880ce930 2024-03-23T07:56:31.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.868+0000 7ff385400640 1 -- 172.21.15.105:0/4226784792 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429561722 0 0) 0x7ff3880cdba0 con 0x7ff3880cd3e0 2024-03-23T07:56:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.870+0000 7ff385e00640 1 --2- 172.21.15.105:0/4226784792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3880ce930 0x7ff3880add30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff378009870 tx=0x7ff37802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2100f6590c4190 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff385400640 1 -- 172.21.15.105:0/4226784792 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378030840 con 0x7ff3880ce930 2024-03-23T07:56:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff385400640 1 -- 172.21.15.105:0/4226784792 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3780309a0 con 0x7ff3880ce930 2024-03-23T07:56:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff385400640 1 -- 172.21.15.105:0/4226784792 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378030d30 con 0x7ff3880ce930 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 -- 172.21.15.105:0/4226784792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3880ce930 msgr2=0x7ff3880add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 --2- 172.21.15.105:0/4226784792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3880ce930 0x7ff3880add30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff378009870 tx=0x7ff37802b980 comp rx=0 tx=0).stop 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 -- 172.21.15.105:0/4226784792 shutdown_connections 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 --2- 172.21.15.105:0/4226784792 >> v2:172.21.15.132:3300/0 conn(0x7ff38806ce90 0x7ff38806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 --2- 172.21.15.105:0/4226784792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3880ce930 0x7ff3880add30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 -- 172.21.15.105:0/4226784792 >> 172.21.15.105:0/4226784792 conn(0x7ff38806a0b0 msgr2=0x7ff38806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 -- 172.21.15.105:0/4226784792 shutdown_connections 2024-03-23T07:56:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.871+0000 7ff38d200640 1 -- 172.21.15.105:0/4226784792 wait complete. 2024-03-23T07:56:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.873+0000 7ff38d200640 1 Processor -- start 2024-03-23T07:56:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.873+0000 7ff38d200640 1 -- start start 2024-03-23T07:56:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff38d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38806ce90 0x7ff3880ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff38d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3880cd3e0 0x7ff38814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff38d200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880ce930 0x7ff3880ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff38d200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880a7ea0 con 0x7ff3880ce930 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff38d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880ab5c0 con 0x7ff3880cd3e0 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff38d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38814d810 con 0x7ff38806ce90 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff386800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880ce930 0x7ff3880ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff386800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880ce930 0x7ff3880ac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41316/0 (socket says 172.21.15.105:41316) 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.874+0000 7ff386800640 1 -- 172.21.15.105:0/1209857776 learned_addr learned my addr 172.21.15.105:0/1209857776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff385e00640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38806ce90 0x7ff3880ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff37fe00640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3880cd3e0 0x7ff38814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff386800640 1 -- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38806ce90 msgr2=0x7ff3880ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff386800640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38806ce90 0x7ff3880ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff386800640 1 -- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3880cd3e0 msgr2=0x7ff38814d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff386800640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3880cd3e0 0x7ff38814d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff386800640 1 -- 172.21.15.105:0/1209857776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3780094d0 con 0x7ff3880ce930 2024-03-23T07:56:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff386800640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880ce930 0x7ff3880ac990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff38000efc0 tx=0x7ff38000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.875+0000 7ff37e000640 1 -- 172.21.15.105:0/1209857776 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000ced0 con 0x7ff3880ce930 2024-03-23T07:56:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.876+0000 7ff37e000640 1 -- 172.21.15.105:0/1209857776 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000f040 con 0x7ff3880ce930 2024-03-23T07:56:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.876+0000 7ff37e000640 1 -- 172.21.15.105:0/1209857776 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3800146c0 con 0x7ff3880ce930 2024-03-23T07:56:31.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.876+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3880a7540 con 0x7ff3880ce930 2024-03-23T07:56:31.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.876+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3880a7ac0 con 0x7ff3880ce930 2024-03-23T07:56:31.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.877+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3880a7fe0 con 0x7ff3880ce930 2024-03-23T07:56:31.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.878+0000 7ff37e000640 1 -- 172.21.15.105:0/1209857776 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff380014820 con 0x7ff3880ce930 2024-03-23T07:56:31.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.879+0000 7ff37e000640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36406ac30 0x7ff36406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:31.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.879+0000 7ff37e000640 1 -- 172.21.15.105:0/1209857776 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff38008a250 con 0x7ff3880ce930 2024-03-23T07:56:31.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.879+0000 7ff385e00640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36406ac30 0x7ff36406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:31.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.880+0000 7ff385e00640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36406ac30 0x7ff36406d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff378036000 tx=0x7ff37803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:31.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:31.883+0000 7ff37e000640 1 -- 172.21.15.105:0/1209857776 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff3880a7fe0 con 0x7ff3880ce930 2024-03-23T07:56:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.078+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3880a7fe0 con 0x7ff3880ce930 2024-03-23T07:56:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.079+0000 7ff37e000640 1 -- 172.21.15.105:0/1209857776 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3880a7fe0 con 0x7ff3880ce930 2024-03-23T07:56:32.078 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:32.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.081+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36406ac30 msgr2=0x7ff36406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.081+0000 7ff38d200640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36406ac30 0x7ff36406d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff378036000 tx=0x7ff37803b040 comp rx=0 tx=0).stop 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.081+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880ce930 msgr2=0x7ff3880ac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.081+0000 7ff38d200640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880ce930 0x7ff3880ac990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff38000efc0 tx=0x7ff38000c490 comp rx=0 tx=0).stop 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 shutdown_connections 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880ce930 0x7ff3880ac990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3880cd3e0 0x7ff38814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36406ac30 0x7ff36406d0f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 --2- 172.21.15.105:0/1209857776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38806ce90 0x7ff3880ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 >> 172.21.15.105:0/1209857776 conn(0x7ff38806a0b0 msgr2=0x7ff3880b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 shutdown_connections 2024-03-23T07:56:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.082+0000 7ff38d200640 1 -- 172.21.15.105:0/1209857776 wait complete. 2024-03-23T07:56:32.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:56:32.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.315+0000 7f8d27400640 1 Processor -- start 2024-03-23T07:56:32.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.315+0000 7f8d27400640 1 -- start start 2024-03-23T07:56:32.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d27400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ca7b0 0x7f8d200cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d27400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d200ce930 0x7f8d200cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d27400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d20159c10 con 0x7f8d20159830 2024-03-23T07:56:32.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d27400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2006de50 con 0x7f8d200ce930 2024-03-23T07:56:32.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d27400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200cf300 con 0x7f8d200ca7b0 2024-03-23T07:56:32.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d1fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ca7b0 0x7f8d200cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d25400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d200ce930 0x7f8d200cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d1fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ca7b0 0x7f8d200cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:60416/0 (socket says 172.21.15.105:60416) 2024-03-23T07:56:32.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.317+0000 7f8d1fe00640 1 -- 172.21.15.105:0/144876688 learned_addr learned my addr 172.21.15.105:0/144876688 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:32.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.318+0000 7f8d25400640 1 -- 172.21.15.105:0/144876688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ca7b0 msgr2=0x7f8d200cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:32.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.318+0000 7f8d25400640 1 --2- 172.21.15.105:0/144876688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ca7b0 0x7f8d200cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.318+0000 7f8d25400640 1 -- 172.21.15.105:0/144876688 >> v1:172.21.15.132:6789/0 conn(0x7f8d20159830 legacy=0x7f8d200cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.318+0000 7f8d25400640 1 -- 172.21.15.105:0/144876688 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200cd840 con 0x7f8d200ce930 2024-03-23T07:56:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.320+0000 7f8d25400640 1 --2- 172.21.15.105:0/144876688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ce930 0x7f8d200cedc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d1802bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c943907c407a96c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.320+0000 7f8d1f400640 1 -- 172.21.15.105:0/144876688 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1802cce0 con 0x7f8d200ce930 2024-03-23T07:56:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.320+0000 7f8d1f400640 1 -- 172.21.15.105:0/144876688 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1802ce40 con 0x7f8d200ce930 2024-03-23T07:56:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.320+0000 7f8d1f400640 1 -- 172.21.15.105:0/144876688 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18034660 con 0x7f8d200ce930 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 -- 172.21.15.105:0/144876688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ce930 msgr2=0x7f8d200cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 --2- 172.21.15.105:0/144876688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ce930 0x7f8d200cedc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d1802bf30 comp rx=0 tx=0).stop 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 -- 172.21.15.105:0/144876688 shutdown_connections 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 --2- 172.21.15.105:0/144876688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ce930 0x7f8d200cedc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 --2- 172.21.15.105:0/144876688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ca7b0 0x7f8d200cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 -- 172.21.15.105:0/144876688 >> 172.21.15.105:0/144876688 conn(0x7f8d2006a0b0 msgr2=0x7f8d2006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 -- 172.21.15.105:0/144876688 shutdown_connections 2024-03-23T07:56:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.321+0000 7f8d27400640 1 -- 172.21.15.105:0/144876688 wait complete. 2024-03-23T07:56:32.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.323+0000 7f8d27400640 1 Processor -- start 2024-03-23T07:56:32.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.323+0000 7f8d27400640 1 -- start start 2024-03-23T07:56:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d27400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ca7b0 0x7f8d200b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d27400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ce930 0x7f8d2014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d27400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20159830 0x7f8d201f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d27400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2015a0b0 con 0x7f8d20159830 2024-03-23T07:56:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d27400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2015a300 con 0x7f8d200ca7b0 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d27400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2015a5e0 con 0x7f8d200ce930 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d25400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20159830 0x7f8d201f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d25400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20159830 0x7f8d201f24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41330/0 (socket says 172.21.15.105:41330) 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d25400640 1 -- 172.21.15.105:0/4206353717 learned_addr learned my addr 172.21.15.105:0/4206353717 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.324+0000 7f8d1fe00640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ce930 0x7f8d2014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d24a00640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ca7b0 0x7f8d200b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d25400640 1 -- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ce930 msgr2=0x7f8d2014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d25400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ce930 0x7f8d2014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d25400640 1 -- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ca7b0 msgr2=0x7f8d200b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d25400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ca7b0 0x7f8d200b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d25400640 1 -- 172.21.15.105:0/4206353717 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d18009a70 con 0x7f8d20159830 2024-03-23T07:56:32.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d25400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20159830 0x7f8d201f24f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:32.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d1cc00640 1 -- 172.21.15.105:0/4206353717 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1803d600 con 0x7f8d20159830 2024-03-23T07:56:32.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d1cc00640 1 -- 172.21.15.105:0/4206353717 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1803d780 con 0x7f8d20159830 2024-03-23T07:56:32.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2015a8c0 con 0x7f8d20159830 2024-03-23T07:56:32.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d1cc00640 1 -- 172.21.15.105:0/4206353717 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1803db30 con 0x7f8d20159830 2024-03-23T07:56:32.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.325+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2015ae80 con 0x7f8d20159830 2024-03-23T07:56:32.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.327+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2015b070 con 0x7f8d20159830 2024-03-23T07:56:32.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.328+0000 7f8d1cc00640 1 -- 172.21.15.105:0/4206353717 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d1803b040 con 0x7f8d20159830 2024-03-23T07:56:32.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.328+0000 7f8d1cc00640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8cfc06ac00 0x7f8cfc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.329+0000 7f8d1cc00640 1 -- 172.21.15.105:0/4206353717 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8d180abc50 con 0x7f8d20159830 2024-03-23T07:56:32.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.329+0000 7f8d24a00640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8cfc06ac00 0x7f8cfc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.329+0000 7f8d24a00640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8cfc06ac00 0x7f8cfc06d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8d10000f60 tx=0x7f8d10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:32.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.333+0000 7f8d1cc00640 1 -- 172.21.15.105:0/4206353717 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d18076bc0 con 0x7f8d20159830 2024-03-23T07:56:32.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.526+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d20073390 con 0x7f8d20159830 2024-03-23T07:56:32.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.527+0000 7f8d1cc00640 1 -- 172.21.15.105:0/4206353717 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d18039020 con 0x7f8d20159830 2024-03-23T07:56:32.527 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:56:32.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8cfc06ac00 msgr2=0x7f8cfc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8cfc06ac00 0x7f8cfc06d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8d10000f60 tx=0x7f8d10009040 comp rx=0 tx=0).stop 2024-03-23T07:56:32.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20159830 msgr2=0x7f8d201f24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20159830 0x7f8d201f24f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d18004a40 comp rx=0 tx=0).stop 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 shutdown_connections 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20159830 0x7f8d201f24f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8cfc06ac00 0x7f8cfc06d0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d200ce930 0x7f8d2014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.530+0000 7f8d27400640 1 --2- 172.21.15.105:0/4206353717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d200ca7b0 0x7f8d200b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.531+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 >> 172.21.15.105:0/4206353717 conn(0x7f8d2006a0b0 msgr2=0x7f8d2006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.531+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 shutdown_connections 2024-03-23T07:56:32.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.531+0000 7f8d27400640 1 -- 172.21.15.105:0/4206353717 wait complete. 2024-03-23T07:56:32.549 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:56:32.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.754+0000 7fb2c2c00640 1 Processor -- start 2024-03-23T07:56:32.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.755+0000 7fb2c2c00640 1 -- start start 2024-03-23T07:56:32.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c2c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c2c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb2bc0ad7a0 0x7fb2bc15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c2c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0c8c90 con 0x7fb2bc0cc200 2024-03-23T07:56:32.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c2c00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0cc6c0 con 0x7fb2bc0ad7a0 2024-03-23T07:56:32.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c2c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0cc9a0 con 0x7fb2bc159cd0 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c2200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb2bc0ad7a0 0x7fb2bc15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c0e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb2bc0cc200 0x7fb2bc0cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43638/0 (socket says 172.21.15.105:43638) 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.756+0000 7fb2c0e00640 1 -- 172.21.15.105:0/3775009337 learned_addr learned my addr 172.21.15.105:0/3775009337 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.757+0000 7fb2c1800640 1 --2- 172.21.15.105:0/3775009337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.757+0000 7fb2c2200640 1 -- 172.21.15.105:0/3775009337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 msgr2=0x7fb2bc0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.757+0000 7fb2c2200640 1 --2- 172.21.15.105:0/3775009337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc0ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.757+0000 7fb2c2200640 1 -- 172.21.15.105:0/3775009337 >> v1:172.21.15.105:6789/0 conn(0x7fb2bc0cc200 legacy=0x7fb2bc0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.757+0000 7fb2c2200640 1 -- 172.21.15.105:0/3775009337 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0ccc80 con 0x7fb2bc0ad7a0 2024-03-23T07:56:32.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.758+0000 7fb2c2200640 1 --2- 172.21.15.105:0/3775009337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0ad7a0 0x7fb2bc15a0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400bfa0 tx=0x7fb2b402e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85f63bdbba02b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:32.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.758+0000 7fb2bbe00640 1 -- 172.21.15.105:0/3775009337 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b402ea40 con 0x7fb2bc0ad7a0 2024-03-23T07:56:32.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.758+0000 7fb2bbe00640 1 -- 172.21.15.105:0/3775009337 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b402eba0 con 0x7fb2bc0ad7a0 2024-03-23T07:56:32.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.758+0000 7fb2bbe00640 1 -- 172.21.15.105:0/3775009337 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4009540 con 0x7fb2bc0ad7a0 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 -- 172.21.15.105:0/3775009337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0ad7a0 msgr2=0x7fb2bc15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/3775009337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0ad7a0 0x7fb2bc15a0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400bfa0 tx=0x7fb2b402e0a0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 -- 172.21.15.105:0/3775009337 shutdown_connections 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/3775009337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0ad7a0 0x7fb2bc15a0b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/3775009337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 -- 172.21.15.105:0/3775009337 >> 172.21.15.105:0/3775009337 conn(0x7fb2bc0b01b0 msgr2=0x7fb2bc0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 -- 172.21.15.105:0/3775009337 shutdown_connections 2024-03-23T07:56:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.759+0000 7fb2c2c00640 1 -- 172.21.15.105:0/3775009337 wait complete. 2024-03-23T07:56:32.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.761+0000 7fb2c2c00640 1 Processor -- start 2024-03-23T07:56:32.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.761+0000 7fb2c2c00640 1 -- start start 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c2c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2bc0ad7a0 0x7fb2bc1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c2c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0cc200 0x7fb2bc0996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c2c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c2c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06cc50 con 0x7fb2bc0cc200 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c2c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06c040 con 0x7fb2bc0ad7a0 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c2c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06c320 con 0x7fb2bc159cd0 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0cc200 0x7fb2bc0996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0cc200 0x7fb2bc0996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41346/0 (socket says 172.21.15.105:41346) 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 -- 172.21.15.105:0/228337220 learned_addr learned my addr 172.21.15.105:0/228337220 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 -- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 msgr2=0x7fb2bc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c2200640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c1800640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2bc0ad7a0 0x7fb2bc1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 -- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2bc0ad7a0 msgr2=0x7fb2bc1f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2bc0ad7a0 0x7fb2bc1f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.762+0000 7fb2c0e00640 1 -- 172.21.15.105:0/228337220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b400bc80 con 0x7fb2bc0cc200 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.763+0000 7fb2c0e00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0cc200 0x7fb2bc0996c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800c4a0 tx=0x7fb2a800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.763+0000 7fb2b9600640 1 -- 172.21.15.105:0/228337220 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8005ce0 con 0x7fb2bc0cc200 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.763+0000 7fb2b9600640 1 -- 172.21.15.105:0/228337220 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a8005e40 con 0x7fb2bc0cc200 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.763+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc06c600 con 0x7fb2bc0cc200 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.763+0000 7fb2b9600640 1 -- 172.21.15.105:0/228337220 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8007270 con 0x7fb2bc0cc200 2024-03-23T07:56:32.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.763+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc1f2580 con 0x7fb2bc0cc200 2024-03-23T07:56:32.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.764+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc04e030 con 0x7fb2bc0cc200 2024-03-23T07:56:32.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.765+0000 7fb2b9600640 1 -- 172.21.15.105:0/228337220 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb2a8007490 con 0x7fb2bc0cc200 2024-03-23T07:56:32.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.766+0000 7fb2b9600640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb29c06ac00 0x7fb29c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:32.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.766+0000 7fb2b9600640 1 -- 172.21.15.105:0/228337220 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb2a8088b30 con 0x7fb2bc0cc200 2024-03-23T07:56:32.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.766+0000 7fb2c1800640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb29c06ac00 0x7fb29c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:32.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.766+0000 7fb2c1800640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb29c06ac00 0x7fb29c06d0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0006fd0 tx=0x7fb2b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:32.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.770+0000 7fb2b9600640 1 -- 172.21.15.105:0/228337220 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb2a8053a30 con 0x7fb2bc0cc200 2024-03-23T07:56:32.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.963+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2bc0cdd20 con 0x7fb2bc0cc200 2024-03-23T07:56:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.964+0000 7fb2b9600640 1 -- 172.21.15.105:0/228337220 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb2a8017020 con 0x7fb2bc0cc200 2024-03-23T07:56:32.964 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb29c06ac00 msgr2=0x7fb29c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb29c06ac00 0x7fb29c06d0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0006fd0 tx=0x7fb2b0008040 comp rx=0 tx=0).stop 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0cc200 msgr2=0x7fb2bc0996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0cc200 0x7fb2bc0996c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800c4a0 tx=0x7fb2a800c960 comp rx=0 tx=0).stop 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 shutdown_connections 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb29c06ac00 0x7fb29c06d0c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2bc159cd0 0x7fb2bc15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2bc0cc200 0x7fb2bc0996c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 --2- 172.21.15.105:0/228337220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2bc0ad7a0 0x7fb2bc1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 >> 172.21.15.105:0/228337220 conn(0x7fb2bc0b01b0 msgr2=0x7fb2bc0ab6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 shutdown_connections 2024-03-23T07:56:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:32.967+0000 7fb2c2c00640 1 -- 172.21.15.105:0/228337220 wait complete. 2024-03-23T07:56:32.985 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T07:56:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.200+0000 7f0bc1200640 1 Processor -- start 2024-03-23T07:56:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.200+0000 7f0bc1200640 1 -- start start 2024-03-23T07:56:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bc1200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0bbc159690 0x7f0bbc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bc1200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc15e010 con 0x7f0bbc0cc330 2024-03-23T07:56:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bc1200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc0cc7f0 con 0x7f0bbc0cd890 2024-03-23T07:56:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bc1200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc0ccad0 con 0x7f0bbc159690 2024-03-23T07:56:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bb9400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0bbc0cc330 0x7f0bbc0b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43654/0 (socket says 172.21.15.105:43654) 2024-03-23T07:56:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bb9400640 1 -- 172.21.15.105:0/3742750027 learned_addr learned my addr 172.21.15.105:0/3742750027 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bb9e00640 1 --2- 172.21.15.105:0/3742750027 >> v2:172.21.15.132:3300/0 conn(0x7f0bbc159690 0x7f0bbc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833313985 0 0) 0x7f0bbc15e010 con 0x7f0bbc0cc330 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.202+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba0003520 con 0x7f0bbc0cc330 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784509058 0 0) 0x7f0ba0003520 con 0x7f0bbc0cc330 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 >> v2:172.21.15.132:3300/0 conn(0x7f0bbc159690 msgr2=0x7f0bbc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 --2- 172.21.15.105:0/3742750027 >> v2:172.21.15.132:3300/0 conn(0x7f0bbc159690 0x7f0bbc0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 >> v1:172.21.15.132:6789/0 conn(0x7f0bbc0cd890 legacy=0x7f0bbc15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc0ccdb0 con 0x7f0bbc0cc330 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f0ba8001670 con 0x7f0bbc0cc330 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 >> v1:172.21.15.105:6789/0 conn(0x7f0bbc0cc330 legacy=0x7f0bbc0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0003970 0x7f0ba0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ba00047c0 0x7f0ba0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba0005180 0x7f0ba000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0ccad0 con 0x7f0ba0005180 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba000e230 con 0x7f0ba00047c0 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba000e3f0 con 0x7f0ba0003970 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.203+0000 7f0bba800640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba0005180 0x7f0ba000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bb9e00640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0003970 0x7f0ba0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bb9400640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ba00047c0 0x7f0ba0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bba800640 1 -- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0003970 msgr2=0x7f0ba0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bba800640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0003970 0x7f0ba0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bba800640 1 -- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ba00047c0 msgr2=0x7f0ba0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bba800640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ba00047c0 0x7f0ba0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bba800640 1 -- 172.21.15.105:0/3742750027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba0003520 con 0x7f0ba0005180 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.204+0000 7f0bb9e00640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0003970 0x7f0ba0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.205+0000 7f0bba800640 1 -- 172.21.15.105:0/3742750027 reap_dead start 2024-03-23T07:56:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.205+0000 7f0bba800640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba0005180 0x7f0ba000db90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4001e50 tx=0x7f0ba00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf6f715c9d78500 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.205+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4003b70 con 0x7f0ba0005180 2024-03-23T07:56:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.205+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb4003d10 con 0x7f0ba0005180 2024-03-23T07:56:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.205+0000 7f0bb8a00640 1 -- 172.21.15.105:0/3742750027 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4005310 con 0x7f0ba0005180 2024-03-23T07:56:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.206+0000 7f0bc1200640 1 -- 172.21.15.105:0/3742750027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba0005180 msgr2=0x7f0ba000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.206+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba0005180 0x7f0ba000db90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4001e50 tx=0x7f0ba00057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.206+0000 7f0bc1200640 1 -- 172.21.15.105:0/3742750027 shutdown_connections 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.206+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3742750027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba0005180 0x7f0ba000db90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.206+0000 7f0bc1200640 1 -- 172.21.15.105:0/3742750027 >> 172.21.15.105:0/3742750027 conn(0x7f0bbc06a0b0 msgr2=0x7f0bbc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.206+0000 7f0bc1200640 1 -- 172.21.15.105:0/3742750027 shutdown_connections 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.206+0000 7f0bc1200640 1 -- 172.21.15.105:0/3742750027 wait complete. 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.208+0000 7f0bc1200640 1 Processor -- start 2024-03-23T07:56:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.208+0000 7f0bc1200640 1 -- start start 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bc1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0005180 0x7f0bbc14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bc1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc134e70 0x7f0bbc14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bc1200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bbc1352d0 0x7f0bbc157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bc1200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc14aa40 con 0x7f0bbc134e70 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bc1200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc14ad20 con 0x7f0bbc1352d0 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bc1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc14b000 con 0x7f0ba0005180 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc134e70 0x7f0bbc14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc134e70 0x7f0bbc14a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41368/0 (socket says 172.21.15.105:41368) 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 -- 172.21.15.105:0/3726197172 learned_addr learned my addr 172.21.15.105:0/3726197172 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9e00640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0005180 0x7f0bbc14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 -- 172.21.15.105:0/3726197172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0005180 msgr2=0x7f0bbc14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0005180 0x7f0bbc14bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 -- 172.21.15.105:0/3726197172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bbc1352d0 msgr2=0x7f0bbc157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bbc1352d0 0x7f0bbc157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.209+0000 7f0bb9400640 1 -- 172.21.15.105:0/3726197172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc14b2e0 con 0x7f0bbc134e70 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.210+0000 7f0bb9400640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc134e70 0x7f0bbc14a080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800e700 tx=0x7f0ba800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.210+0000 7f0bb2000640 1 -- 172.21.15.105:0/3726197172 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800c6c0 con 0x7f0bbc134e70 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.210+0000 7f0bb2000640 1 -- 172.21.15.105:0/3726197172 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba800c840 con 0x7f0bbc134e70 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.210+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc14b5c0 con 0x7f0bbc134e70 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.210+0000 7f0bb2000640 1 -- 172.21.15.105:0/3726197172 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800cbf0 con 0x7f0bbc134e70 2024-03-23T07:56:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.210+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc13c420 con 0x7f0bbc134e70 2024-03-23T07:56:33.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.211+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bbc073390 con 0x7f0bbc134e70 2024-03-23T07:56:33.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.212+0000 7f0bb2000640 1 -- 172.21.15.105:0/3726197172 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0ba800cd50 con 0x7f0bbc134e70 2024-03-23T07:56:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.213+0000 7f0bb2000640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b9c06ac00 0x7f0b9c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.214+0000 7f0bb2000640 1 -- 172.21.15.105:0/3726197172 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0ba8089620 con 0x7f0bbc134e70 2024-03-23T07:56:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.214+0000 7f0bb9e00640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b9c06ac00 0x7f0b9c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.215+0000 7f0bb9e00640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b9c06ac00 0x7f0b9c06d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0099c0 tx=0x7f0bac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:33.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.218+0000 7f0bb2000640 1 -- 172.21.15.105:0/3726197172 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0ba8054480 con 0x7f0bbc134e70 2024-03-23T07:56:33.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.425+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0bbc0cf230 con 0x7f0bbc134e70 2024-03-23T07:56:33.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.425+0000 7f0bb2000640 1 -- 172.21.15.105:0/3726197172 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0ba8057aa0 con 0x7f0bbc134e70 2024-03-23T07:56:33.425 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-23T07:56:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b9c06ac00 msgr2=0x7f0b9c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b9c06ac00 0x7f0b9c06d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0099c0 tx=0x7f0bac008040 comp rx=0 tx=0).stop 2024-03-23T07:56:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc134e70 msgr2=0x7f0bbc14a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc134e70 0x7f0bbc14a080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800e700 tx=0x7f0ba800ebc0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 shutdown_connections 2024-03-23T07:56:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bbc1352d0 0x7f0bbc157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc134e70 0x7f0bbc14a080 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b9c06ac00 0x7f0b9c06d0c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 --2- 172.21.15.105:0/3726197172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ba0005180 0x7f0bbc14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.428+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 >> 172.21.15.105:0/3726197172 conn(0x7f0bbc06a0b0 msgr2=0x7f0bbc154100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:33.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.429+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 shutdown_connections 2024-03-23T07:56:33.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.429+0000 7f0bc1200640 1 -- 172.21.15.105:0/3726197172 wait complete. 2024-03-23T07:56:33.444 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:56:33.445 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:56:33.491 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.746+0000 7fa37aa00640 1 Processor -- start 2024-03-23T07:56:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.746+0000 7fa37aa00640 1 -- start start 2024-03-23T07:56:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.747+0000 7fa37aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740adbc0 0x7fa3740b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.747+0000 7fa37aa00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa3740cbab0 0x7fa3740b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa37aa00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3740b16a0 con 0x7fa37406d390 2024-03-23T07:56:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa37aa00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740b4700 con 0x7fa3740cbab0 2024-03-23T07:56:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa37aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740cbfb0 con 0x7fa3740adbc0 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa373400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa3740cbab0 0x7fa3740b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa373400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa3740cbab0 0x7fa3740b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41378/0 (socket says 172.21.15.105:41378) 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa373400640 1 -- 172.21.15.105:0/1945407024 learned_addr learned my addr 172.21.15.105:0/1945407024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa373e00640 1 --2- 172.21.15.105:0/1945407024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740adbc0 0x7fa3740b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa373400640 1 -- 172.21.15.105:0/1945407024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740adbc0 msgr2=0x7fa3740b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.748+0000 7fa373400640 1 --2- 172.21.15.105:0/1945407024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740adbc0 0x7fa3740b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.749+0000 7fa373400640 1 -- 172.21.15.105:0/1945407024 >> v1:172.21.15.132:6789/0 conn(0x7fa37406d390 legacy=0x7fa3740b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.749+0000 7fa373400640 1 -- 172.21.15.105:0/1945407024 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740cc290 con 0x7fa3740cbab0 2024-03-23T07:56:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.750+0000 7fa373400640 1 --2- 172.21.15.105:0/1945407024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740cbab0 0x7fa3740b20f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa360009870 tx=0x7fa36002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3121d67a84bcf00 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.750+0000 7fa372a00640 1 -- 172.21.15.105:0/1945407024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360031640 con 0x7fa3740cbab0 2024-03-23T07:56:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.750+0000 7fa372a00640 1 -- 172.21.15.105:0/1945407024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3600317a0 con 0x7fa3740cbab0 2024-03-23T07:56:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.750+0000 7fa372a00640 1 -- 172.21.15.105:0/1945407024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360031b30 con 0x7fa3740cbab0 2024-03-23T07:56:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 -- 172.21.15.105:0/1945407024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740cbab0 msgr2=0x7fa3740b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1945407024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740cbab0 0x7fa3740b20f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa360009870 tx=0x7fa36002b8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 -- 172.21.15.105:0/1945407024 shutdown_connections 2024-03-23T07:56:33.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1945407024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740cbab0 0x7fa3740b20f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1945407024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740adbc0 0x7fa3740b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 -- 172.21.15.105:0/1945407024 >> 172.21.15.105:0/1945407024 conn(0x7fa37406a3d0 msgr2=0x7fa3740b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:33.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 -- 172.21.15.105:0/1945407024 shutdown_connections 2024-03-23T07:56:33.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.751+0000 7fa37aa00640 1 -- 172.21.15.105:0/1945407024 wait complete. 2024-03-23T07:56:33.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.753+0000 7fa37aa00640 1 Processor -- start 2024-03-23T07:56:33.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.753+0000 7fa37aa00640 1 -- start start 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa37aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa37406d390 0x7fa37413bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa37aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740adbc0 0x7fa3740a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa37aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740cbab0 0x7fa3740a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa37aa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740a79e0 con 0x7fa3740adbc0 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa37aa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740aa410 con 0x7fa37406d390 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa37aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740a7bf0 con 0x7fa3740cbab0 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740adbc0 0x7fa3740a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740adbc0 0x7fa3740a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41380/0 (socket says 172.21.15.105:41380) 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 -- 172.21.15.105:0/1414740967 learned_addr learned my addr 172.21.15.105:0/1414740967 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373e00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa37406d390 0x7fa37413bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa378a00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740cbab0 0x7fa3740a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 -- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740cbab0 msgr2=0x7fa3740a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740cbab0 0x7fa3740a9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 -- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa37406d390 msgr2=0x7fa37413bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa37406d390 0x7fa37413bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.754+0000 7fa373400640 1 -- 172.21.15.105:0/1414740967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600094d0 con 0x7fa3740adbc0 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.755+0000 7fa373400640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740adbc0 0x7fa3740a74a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa360000c00 tx=0x7fa3600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.755+0000 7fa36be00640 1 -- 172.21.15.105:0/1414740967 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36003ac60 con 0x7fa3740adbc0 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.755+0000 7fa36be00640 1 -- 172.21.15.105:0/1414740967 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36003adc0 con 0x7fa3740adbc0 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.755+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3740a7ed0 con 0x7fa3740adbc0 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.755+0000 7fa36be00640 1 -- 172.21.15.105:0/1414740967 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360044630 con 0x7fa3740adbc0 2024-03-23T07:56:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.755+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3740a8490 con 0x7fa3740adbc0 2024-03-23T07:56:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.756+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37406ac20 con 0x7fa3740adbc0 2024-03-23T07:56:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.757+0000 7fa36be00640 1 -- 172.21.15.105:0/1414740967 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa36002f050 con 0x7fa3740adbc0 2024-03-23T07:56:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.758+0000 7fa36be00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa35406ac00 0x7fa35406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.758+0000 7fa36be00640 1 -- 172.21.15.105:0/1414740967 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa3600ab540 con 0x7fa3740adbc0 2024-03-23T07:56:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.758+0000 7fa373e00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa35406ac00 0x7fa35406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.759+0000 7fa373e00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa35406ac00 0x7fa35406d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa364006fd0 tx=0x7fa364008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.763+0000 7fa36be00640 1 -- 172.21.15.105:0/1414740967 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa360076420 con 0x7fa3740adbc0 2024-03-23T07:56:33.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.958+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa374073390 con 0x7fa3740adbc0 2024-03-23T07:56:33.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.962+0000 7fa36be00640 1 -- 172.21.15.105:0/1414740967 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7fa360031e60 con 0x7fa3740adbc0 2024-03-23T07:56:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:33.962 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:33.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.964+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa35406ac00 msgr2=0x7fa35406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.964+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa35406ac00 0x7fa35406d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa364006fd0 tx=0x7fa364008040 comp rx=0 tx=0).stop 2024-03-23T07:56:33.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.964+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740adbc0 msgr2=0x7fa3740a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.964+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740adbc0 0x7fa3740a74a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa360000c00 tx=0x7fa3600046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 shutdown_connections 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa35406ac00 0x7fa35406d0c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3740cbab0 0x7fa3740a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740adbc0 0x7fa3740a74a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 --2- 172.21.15.105:0/1414740967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa37406d390 0x7fa37413bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 >> 172.21.15.105:0/1414740967 conn(0x7fa37406a3d0 msgr2=0x7fa374142bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 shutdown_connections 2024-03-23T07:56:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:33.965+0000 7fa37aa00640 1 -- 172.21.15.105:0/1414740967 wait complete. 2024-03-23T07:56:33.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:56:33.982 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:56:34.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.198+0000 7f59e0e00640 1 Processor -- start 2024-03-23T07:56:34.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.198+0000 7f59e0e00640 1 -- start start 2024-03-23T07:56:34.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.199+0000 7f59e0e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f59dc159830 0x7f59dc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.199+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59e0e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc159c10 con 0x7f59dc0ce930 2024-03-23T07:56:34.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59e0e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc06de50 con 0x7f59dc159830 2024-03-23T07:56:34.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59e0e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0cd840 con 0x7f59dc0ca7b0 2024-03-23T07:56:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59d3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59d9e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f59dc159830 0x7f59dc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59d3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:60494/0 (socket says 172.21.15.105:60494) 2024-03-23T07:56:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59d9e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f59dc159830 0x7f59dc0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:35002/0 (socket says 172.21.15.105:35002) 2024-03-23T07:56:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.200+0000 7f59d3e00640 1 -- 172.21.15.105:0/1553672368 learned_addr learned my addr 172.21.15.105:0/1553672368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:34.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.201+0000 7f59d3e00640 1 -- 172.21.15.105:0/1553672368 >> v2:172.21.15.132:3300/0 conn(0x7f59dc159830 msgr2=0x7f59dc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:34.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.201+0000 7f59d3e00640 1 --2- 172.21.15.105:0/1553672368 >> v2:172.21.15.132:3300/0 conn(0x7f59dc159830 0x7f59dc0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.201+0000 7f59d3e00640 1 -- 172.21.15.105:0/1553672368 >> v1:172.21.15.132:6789/0 conn(0x7f59dc0ce930 legacy=0x7f59dc0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.201+0000 7f59d3e00640 1 -- 172.21.15.105:0/1553672368 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc0ccc30 con 0x7f59dc0ca7b0 2024-03-23T07:56:34.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.201+0000 7f59d9e00640 1 --2- 172.21.15.105:0/1553672368 >> v2:172.21.15.132:3300/0 conn(0x7f59dc159830 0x7f59dc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:34.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.203+0000 7f59d3e00640 1 --2- 172.21.15.105:0/1553672368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0cdcd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009870 tx=0x7f59c402dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d379abf86447221 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:34.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.203+0000 7f59d9400640 1 -- 172.21.15.105:0/1553672368 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c402e9e0 con 0x7f59dc0ca7b0 2024-03-23T07:56:34.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59d9400640 1 -- 172.21.15.105:0/1553672368 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c402eb40 con 0x7f59dc0ca7b0 2024-03-23T07:56:34.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59d9400640 1 -- 172.21.15.105:0/1553672368 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4036440 con 0x7f59dc0ca7b0 2024-03-23T07:56:34.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59e0e00640 1 -- 172.21.15.105:0/1553672368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 msgr2=0x7f59dc0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59e0e00640 1 --2- 172.21.15.105:0/1553672368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0cdcd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009870 tx=0x7f59c402dc30 comp rx=0 tx=0).stop 2024-03-23T07:56:34.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59e0e00640 1 -- 172.21.15.105:0/1553672368 shutdown_connections 2024-03-23T07:56:34.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59e0e00640 1 --2- 172.21.15.105:0/1553672368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0cdcd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59e0e00640 1 --2- 172.21.15.105:0/1553672368 >> v2:172.21.15.132:3300/0 conn(0x7f59dc159830 0x7f59dc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.204+0000 7f59e0e00640 1 -- 172.21.15.105:0/1553672368 >> 172.21.15.105:0/1553672368 conn(0x7f59dc06a0b0 msgr2=0x7f59dc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:34.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.205+0000 7f59e0e00640 1 -- 172.21.15.105:0/1553672368 shutdown_connections 2024-03-23T07:56:34.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.205+0000 7f59e0e00640 1 -- 172.21.15.105:0/1553672368 wait complete. 2024-03-23T07:56:34.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.206+0000 7f59e0e00640 1 Processor -- start 2024-03-23T07:56:34.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.206+0000 7f59e0e00640 1 -- start start 2024-03-23T07:56:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.207+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.207+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59dc0ce930 0x7f59dc14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.207+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc159830 0x7f59dc0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.207+0000 7f59e0e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0aefc0 con 0x7f59dc159830 2024-03-23T07:56:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.207+0000 7f59e0e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc14e770 con 0x7f59dc0ce930 2024-03-23T07:56:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.207+0000 7f59e0e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc14fe80 con 0x7f59dc0ca7b0 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc159830 0x7f59dc0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc159830 0x7f59dc0b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41396/0 (socket says 172.21.15.105:41396) 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 -- 172.21.15.105:0/3499723443 learned_addr learned my addr 172.21.15.105:0/3499723443 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59d9e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59d3e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59dc0ce930 0x7f59dc14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 -- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 msgr2=0x7f59dc0ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 -- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59dc0ce930 msgr2=0x7f59dc14e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59dc0ce930 0x7f59dc14e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 -- 172.21.15.105:0/3499723443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc009400 con 0x7f59dc159830 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59d3e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59dc0ce930 0x7f59dc14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59d9e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.208+0000 7f59da800640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc159830 0x7f59dc0b0070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c7a0 tx=0x7f59d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:34.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.209+0000 7f59d2000640 1 -- 172.21.15.105:0/3499723443 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4003b70 con 0x7f59dc159830 2024-03-23T07:56:34.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.209+0000 7f59d2000640 1 -- 172.21.15.105:0/3499723443 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4003cf0 con 0x7f59dc159830 2024-03-23T07:56:34.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.209+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c40094d0 con 0x7f59dc159830 2024-03-23T07:56:34.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.209+0000 7f59d2000640 1 -- 172.21.15.105:0/3499723443 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d400aa90 con 0x7f59dc159830 2024-03-23T07:56:34.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.209+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc0ae720 con 0x7f59dc159830 2024-03-23T07:56:34.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.210+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc150300 con 0x7f59dc159830 2024-03-23T07:56:34.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.211+0000 7f59d2000640 1 -- 172.21.15.105:0/3499723443 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f59d400abf0 con 0x7f59dc159830 2024-03-23T07:56:34.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.212+0000 7f59d2000640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59b806ac00 0x7f59b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.212+0000 7f59d2000640 1 -- 172.21.15.105:0/3499723443 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59d4089630 con 0x7f59dc159830 2024-03-23T07:56:34.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.212+0000 7f59d9e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59b806ac00 0x7f59b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.213+0000 7f59d9e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59b806ac00 0x7f59b806d0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009bb0 tx=0x7f59cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.217+0000 7f59d2000640 1 -- 172.21.15.105:0/3499723443 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f59d4054400 con 0x7f59dc159830 2024-03-23T07:56:34.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.412+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59dc073390 con 0x7f59dc159830 2024-03-23T07:56:34.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.413+0000 7f59d2000640 1 -- 172.21.15.105:0/3499723443 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59d4003e60 con 0x7f59dc159830 2024-03-23T07:56:34.413 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59b806ac00 msgr2=0x7f59b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59b806ac00 0x7f59b806d0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009bb0 tx=0x7f59cc008040 comp rx=0 tx=0).stop 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc159830 msgr2=0x7f59dc0b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc159830 0x7f59dc0b0070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c7a0 tx=0x7f59d400cc60 comp rx=0 tx=0).stop 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 shutdown_connections 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc159830 0x7f59dc0b0070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59dc0ce930 0x7f59dc14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59b806ac00 0x7f59b806d0c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 --2- 172.21.15.105:0/3499723443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59dc0ca7b0 0x7f59dc0ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 >> 172.21.15.105:0/3499723443 conn(0x7f59dc06a0b0 msgr2=0x7f59dc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:34.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 shutdown_connections 2024-03-23T07:56:34.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.416+0000 7f59e0e00640 1 -- 172.21.15.105:0/3499723443 wait complete. 2024-03-23T07:56:34.434 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:56:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.639+0000 7fc486a00640 1 Processor -- start 2024-03-23T07:56:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.639+0000 7fc486a00640 1 -- start start 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.640+0000 7fc486a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc4800ce930 0x7fc4800add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc486a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc48006ce90 0x7fc48006d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc486a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4800cdba0 con 0x7fc4800cd3e0 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc486a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48006c760 con 0x7fc48006ce90 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc486a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4800b42d0 con 0x7fc4800ce930 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc484a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc48006ce90 0x7fc48006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc484a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc48006ce90 0x7fc48006d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41402/0 (socket says 172.21.15.105:41402) 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc484a00640 1 -- 172.21.15.105:0/1370838373 learned_addr learned my addr 172.21.15.105:0/1370838373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:34.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc47fe00640 1 --2- 172.21.15.105:0/1370838373 >> v2:172.21.15.132:3300/0 conn(0x7fc4800ce930 0x7fc4800add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc484a00640 1 -- 172.21.15.105:0/1370838373 >> v2:172.21.15.132:3300/0 conn(0x7fc4800ce930 msgr2=0x7fc4800add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc484a00640 1 --2- 172.21.15.105:0/1370838373 >> v2:172.21.15.132:3300/0 conn(0x7fc4800ce930 0x7fc4800add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc484a00640 1 -- 172.21.15.105:0/1370838373 >> v1:172.21.15.132:6789/0 conn(0x7fc4800cd3e0 legacy=0x7fc4800ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.641+0000 7fc484a00640 1 -- 172.21.15.105:0/1370838373 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800cde80 con 0x7fc48006ce90 2024-03-23T07:56:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.642+0000 7fc484a00640 1 --2- 172.21.15.105:0/1370838373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48006ce90 0x7fc48006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc4800cf540 tx=0x7fc47802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a8b32818728c89 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.643+0000 7fc47ea00640 1 -- 172.21.15.105:0/1370838373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47802ca80 con 0x7fc48006ce90 2024-03-23T07:56:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.643+0000 7fc47ea00640 1 -- 172.21.15.105:0/1370838373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47802cbe0 con 0x7fc48006ce90 2024-03-23T07:56:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.643+0000 7fc47ea00640 1 -- 172.21.15.105:0/1370838373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780343f0 con 0x7fc48006ce90 2024-03-23T07:56:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 -- 172.21.15.105:0/1370838373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48006ce90 msgr2=0x7fc48006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 --2- 172.21.15.105:0/1370838373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48006ce90 0x7fc48006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc4800cf540 tx=0x7fc47802bee0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 -- 172.21.15.105:0/1370838373 shutdown_connections 2024-03-23T07:56:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 --2- 172.21.15.105:0/1370838373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48006ce90 0x7fc48006d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 --2- 172.21.15.105:0/1370838373 >> v2:172.21.15.132:3300/0 conn(0x7fc4800ce930 0x7fc4800add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 -- 172.21.15.105:0/1370838373 >> 172.21.15.105:0/1370838373 conn(0x7fc48006a0b0 msgr2=0x7fc48006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 -- 172.21.15.105:0/1370838373 shutdown_connections 2024-03-23T07:56:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.644+0000 7fc486a00640 1 -- 172.21.15.105:0/1370838373 wait complete. 2024-03-23T07:56:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.646+0000 7fc486a00640 1 Processor -- start 2024-03-23T07:56:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.646+0000 7fc486a00640 1 -- start start 2024-03-23T07:56:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc486a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48006ce90 0x7fc4800c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc486a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4800cd3e0 0x7fc4801595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc486a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4800ce930 0x7fc480159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc486a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc480144f00 con 0x7fc4800ce930 2024-03-23T07:56:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc486a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc480156730 con 0x7fc4800cd3e0 2024-03-23T07:56:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc486a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc480156a10 con 0x7fc48006ce90 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4800ce930 0x7fc480159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4800ce930 0x7fc480159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41408/0 (socket says 172.21.15.105:41408) 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 -- 172.21.15.105:0/2842196916 learned_addr learned my addr 172.21.15.105:0/2842196916 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc47fe00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48006ce90 0x7fc4800c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc47f400640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4800cd3e0 0x7fc4801595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 -- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48006ce90 msgr2=0x7fc4800c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48006ce90 0x7fc4800c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 -- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4800cd3e0 msgr2=0x7fc4801595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4800cd3e0 0x7fc4801595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc484a00640 1 -- 172.21.15.105:0/2842196916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc478009b30 con 0x7fc4800ce930 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.647+0000 7fc47f400640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4800cd3e0 0x7fc4801595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.648+0000 7fc484a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4800ce930 0x7fc480159b30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc478036040 tx=0x7fc478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.648+0000 7fc477e00640 1 -- 172.21.15.105:0/2842196916 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478009510 con 0x7fc4800ce930 2024-03-23T07:56:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.648+0000 7fc477e00640 1 -- 172.21.15.105:0/2842196916 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47803e450 con 0x7fc4800ce930 2024-03-23T07:56:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.648+0000 7fc477e00640 1 -- 172.21.15.105:0/2842196916 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47803e7e0 con 0x7fc4800ce930 2024-03-23T07:56:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.648+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc480156cf0 con 0x7fc4800ce930 2024-03-23T07:56:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.648+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4800ca330 con 0x7fc4800ce930 2024-03-23T07:56:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.649+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4801450b0 con 0x7fc4800ce930 2024-03-23T07:56:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.650+0000 7fc477e00640 1 -- 172.21.15.105:0/2842196916 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc47802cd50 con 0x7fc4800ce930 2024-03-23T07:56:34.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.651+0000 7fc477e00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46006ac00 0x7fc46006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:34.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.651+0000 7fc477e00640 1 -- 172.21.15.105:0/2842196916 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc4780ac210 con 0x7fc4800ce930 2024-03-23T07:56:34.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.651+0000 7fc47fe00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46006ac00 0x7fc46006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:34.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.652+0000 7fc47fe00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46006ac00 0x7fc46006d0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc470006fd0 tx=0x7fc470008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:34.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.655+0000 7fc477e00640 1 -- 172.21.15.105:0/2842196916 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc4801450b0 con 0x7fc4800ce930 2024-03-23T07:56:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.849+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc480073390 con 0x7fc4800ce930 2024-03-23T07:56:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.849+0000 7fc477e00640 1 -- 172.21.15.105:0/2842196916 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc478077050 con 0x7fc4800ce930 2024-03-23T07:56:34.849 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46006ac00 msgr2=0x7fc46006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46006ac00 0x7fc46006d0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc470006fd0 tx=0x7fc470008040 comp rx=0 tx=0).stop 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4800ce930 msgr2=0x7fc480159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4800ce930 0x7fc480159b30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc478036040 tx=0x7fc478004a40 comp rx=0 tx=0).stop 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 shutdown_connections 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4800ce930 0x7fc480159b30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4800cd3e0 0x7fc4801595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46006ac00 0x7fc46006d0c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 --2- 172.21.15.105:0/2842196916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48006ce90 0x7fc4800c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 >> 172.21.15.105:0/2842196916 conn(0x7fc48006a0b0 msgr2=0x7fc4800cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 shutdown_connections 2024-03-23T07:56:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:34.852+0000 7fc486a00640 1 -- 172.21.15.105:0/2842196916 wait complete. 2024-03-23T07:56:34.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:56:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.084+0000 7f623cc00640 1 Processor -- start 2024-03-23T07:56:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.085+0000 7f623cc00640 1 -- start start 2024-03-23T07:56:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.086+0000 7f623cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f62380b84b0 0x7f62380d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.086+0000 7f623cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.086+0000 7f623cc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62380d1280 con 0x7f62380cd740 2024-03-23T07:56:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.087+0000 7f623cc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62380d20c0 con 0x7f62380b84b0 2024-03-23T07:56:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.087+0000 7f623cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62380b8940 con 0x7f62380b5aa0 2024-03-23T07:56:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.087+0000 7f6235400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f62380b84b0 0x7f62380d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.087+0000 7f6236800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.087+0000 7f6235400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f62380b84b0 0x7f62380d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:35032/0 (socket says 172.21.15.105:35032) 2024-03-23T07:56:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.087+0000 7f6235400640 1 -- 172.21.15.105:0/1624394944 learned_addr learned my addr 172.21.15.105:0/1624394944 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.088+0000 7f6236800640 1 -- 172.21.15.105:0/1624394944 >> v2:172.21.15.132:3300/0 conn(0x7f62380b84b0 msgr2=0x7f62380d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.088+0000 7f6236800640 1 --2- 172.21.15.105:0/1624394944 >> v2:172.21.15.132:3300/0 conn(0x7f62380b84b0 0x7f62380d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.088+0000 7f6236800640 1 -- 172.21.15.105:0/1624394944 >> v1:172.21.15.132:6789/0 conn(0x7f62380cd740 legacy=0x7f62380cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.088+0000 7f6236800640 1 -- 172.21.15.105:0/1624394944 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380d14b0 con 0x7f62380b5aa0 2024-03-23T07:56:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.088+0000 7f6235400640 1 --2- 172.21.15.105:0/1624394944 >> v2:172.21.15.132:3300/0 conn(0x7f62380b84b0 0x7f62380d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.089+0000 7f6236800640 1 --2- 172.21.15.105:0/1624394944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380b5f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f62300097a0 tx=0x7f623002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0485759d99ebdf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.090+0000 7f6234a00640 1 -- 172.21.15.105:0/1624394944 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623002d040 con 0x7f62380b5aa0 2024-03-23T07:56:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.090+0000 7f6234a00640 1 -- 172.21.15.105:0/1624394944 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6230031810 con 0x7f62380b5aa0 2024-03-23T07:56:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.090+0000 7f6234a00640 1 -- 172.21.15.105:0/1624394944 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6230031ba0 con 0x7f62380b5aa0 2024-03-23T07:56:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.090+0000 7f623cc00640 1 -- 172.21.15.105:0/1624394944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 msgr2=0x7f62380b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.090+0000 7f623cc00640 1 --2- 172.21.15.105:0/1624394944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380b5f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f62300097a0 tx=0x7f623002b8b0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.091+0000 7f623cc00640 1 -- 172.21.15.105:0/1624394944 shutdown_connections 2024-03-23T07:56:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.091+0000 7f623cc00640 1 --2- 172.21.15.105:0/1624394944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380b5f30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.091+0000 7f623cc00640 1 --2- 172.21.15.105:0/1624394944 >> v2:172.21.15.132:3300/0 conn(0x7f62380b84b0 0x7f62380d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.091+0000 7f623cc00640 1 -- 172.21.15.105:0/1624394944 >> 172.21.15.105:0/1624394944 conn(0x7f62380cf5d0 msgr2=0x7f62380cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:35.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.091+0000 7f623cc00640 1 -- 172.21.15.105:0/1624394944 shutdown_connections 2024-03-23T07:56:35.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.091+0000 7f623cc00640 1 -- 172.21.15.105:0/1624394944 wait complete. 2024-03-23T07:56:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.093+0000 7f623cc00640 1 Processor -- start 2024-03-23T07:56:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.093+0000 7f623cc00640 1 -- start start 2024-03-23T07:56:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.094+0000 7f623cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.094+0000 7f623cc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62380b84b0 0x7f62381541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.094+0000 7f623cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62380cd740 0x7f6238143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.094+0000 7f623cc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380c8aa0 con 0x7f62380b84b0 2024-03-23T07:56:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.094+0000 7f623cc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380b3f30 con 0x7f62380cd740 2024-03-23T07:56:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.094+0000 7f623cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62380b3320 con 0x7f62380b5aa0 2024-03-23T07:56:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62380b84b0 0x7f62381541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62380b84b0 0x7f62381541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41412/0 (socket says 172.21.15.105:41412) 2024-03-23T07:56:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 -- 172.21.15.105:0/3684310683 learned_addr learned my addr 172.21.15.105:0/3684310683 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6236800640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62380cd740 0x7f6238143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 -- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 msgr2=0x7f62380c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 -- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62380cd740 msgr2=0x7f6238143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62380cd740 0x7f6238143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6235400640 1 -- 172.21.15.105:0/3684310683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228009b30 con 0x7f62380b84b0 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.095+0000 7f6236800640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62380cd740 0x7f6238143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.096+0000 7f6235400640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62380b84b0 0x7f62381541f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f622800efc0 tx=0x7f622800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.096+0000 7f622e000640 1 -- 172.21.15.105:0/3684310683 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228014480 con 0x7f62380b84b0 2024-03-23T07:56:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.096+0000 7f622e000640 1 -- 172.21.15.105:0/3684310683 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622800f040 con 0x7f62380b84b0 2024-03-23T07:56:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.096+0000 7f622e000640 1 -- 172.21.15.105:0/3684310683 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62280147e0 con 0x7f62380b84b0 2024-03-23T07:56:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.096+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6230009400 con 0x7f62380b84b0 2024-03-23T07:56:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.096+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62380b38e0 con 0x7f62380b84b0 2024-03-23T07:56:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.097+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62380b40e0 con 0x7f62380b84b0 2024-03-23T07:56:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.098+0000 7f622e000640 1 -- 172.21.15.105:0/3684310683 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6228014940 con 0x7f62380b84b0 2024-03-23T07:56:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.099+0000 7f622e000640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f621406ac00 0x7f621406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.099+0000 7f622e000640 1 -- 172.21.15.105:0/3684310683 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6228089f70 con 0x7f62380b84b0 2024-03-23T07:56:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.099+0000 7f6235e00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f621406ac00 0x7f621406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.100+0000 7f6235e00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f621406ac00 0x7f621406d0c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f623815a730 tx=0x7f6220009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.104+0000 7f622e000640 1 -- 172.21.15.105:0/3684310683 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6228054dc0 con 0x7f62380b84b0 2024-03-23T07:56:35.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.298+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6238073390 con 0x7f62380b84b0 2024-03-23T07:56:35.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.299+0000 7f622e000640 1 -- 172.21.15.105:0/3684310683 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62280583e0 con 0x7f62380b84b0 2024-03-23T07:56:35.299 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:56:35.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f621406ac00 msgr2=0x7f621406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f621406ac00 0x7f621406d0c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f623815a730 tx=0x7f6220009040 comp rx=0 tx=0).stop 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62380b84b0 msgr2=0x7f62381541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62380b84b0 0x7f62381541f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f622800efc0 tx=0x7f622800c490 comp rx=0 tx=0).stop 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 shutdown_connections 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62380cd740 0x7f6238143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62380b84b0 0x7f62381541f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f621406ac00 0x7f621406d0c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 --2- 172.21.15.105:0/3684310683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62380b5aa0 0x7f62380c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 >> 172.21.15.105:0/3684310683 conn(0x7f62380cf5d0 msgr2=0x7f62380d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:35.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.302+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 shutdown_connections 2024-03-23T07:56:35.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.303+0000 7f623cc00640 1 -- 172.21.15.105:0/3684310683 wait complete. 2024-03-23T07:56:35.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:56:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.535+0000 7f6941800640 1 Processor -- start 2024-03-23T07:56:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.535+0000 7f6941800640 1 -- start start 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f6941800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f693c0ca7b0 0x7f693c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f6941800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f693c0ce930 0x7f693c0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f6941800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693c159c10 con 0x7f693c159830 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f6941800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693c06de50 con 0x7f693c0ce930 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f6941800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693c0cf300 con 0x7f693c0ca7b0 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f693b400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f693c0ce930 0x7f693c0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f693b400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f693c0ce930 0x7f693c0cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41420/0 (socket says 172.21.15.105:41420) 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f693b400640 1 -- 172.21.15.105:0/3778366679 learned_addr learned my addr 172.21.15.105:0/3778366679 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f693a000640 1 --2- 172.21.15.105:0/3778366679 >> v2:172.21.15.132:3300/0 conn(0x7f693c0ca7b0 0x7f693c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f693b400640 1 -- 172.21.15.105:0/3778366679 >> v2:172.21.15.132:3300/0 conn(0x7f693c0ca7b0 msgr2=0x7f693c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f693b400640 1 --2- 172.21.15.105:0/3778366679 >> v2:172.21.15.132:3300/0 conn(0x7f693c0ca7b0 0x7f693c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.537+0000 7f693b400640 1 -- 172.21.15.105:0/3778366679 >> v1:172.21.15.132:6789/0 conn(0x7f693c159830 legacy=0x7f693c0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.538+0000 7f693b400640 1 -- 172.21.15.105:0/3778366679 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c0cd840 con 0x7f693c0ce930 2024-03-23T07:56:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.538+0000 7f693a000640 1 --2- 172.21.15.105:0/3778366679 >> v2:172.21.15.132:3300/0 conn(0x7f693c0ca7b0 0x7f693c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.539+0000 7f693b400640 1 --2- 172.21.15.105:0/3778366679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ce930 0x7f693c0cedc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6934009e80 tx=0x7f693402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213b1b2cd9fe45bf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.539+0000 7f6939600640 1 -- 172.21.15.105:0/3778366679 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693402caf0 con 0x7f693c0ce930 2024-03-23T07:56:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.539+0000 7f6939600640 1 -- 172.21.15.105:0/3778366679 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693402cc50 con 0x7f693c0ce930 2024-03-23T07:56:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.539+0000 7f6939600640 1 -- 172.21.15.105:0/3778366679 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69340343f0 con 0x7f693c0ce930 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 -- 172.21.15.105:0/3778366679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ce930 msgr2=0x7f693c0cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 --2- 172.21.15.105:0/3778366679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ce930 0x7f693c0cedc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6934009e80 tx=0x7f693402bf50 comp rx=0 tx=0).stop 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 -- 172.21.15.105:0/3778366679 shutdown_connections 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 --2- 172.21.15.105:0/3778366679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ce930 0x7f693c0cedc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 --2- 172.21.15.105:0/3778366679 >> v2:172.21.15.132:3300/0 conn(0x7f693c0ca7b0 0x7f693c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 -- 172.21.15.105:0/3778366679 >> 172.21.15.105:0/3778366679 conn(0x7f693c06a0b0 msgr2=0x7f693c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 -- 172.21.15.105:0/3778366679 shutdown_connections 2024-03-23T07:56:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.540+0000 7f6941800640 1 -- 172.21.15.105:0/3778366679 wait complete. 2024-03-23T07:56:35.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.542+0000 7f6941800640 1 Processor -- start 2024-03-23T07:56:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.542+0000 7f6941800640 1 -- start start 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f6941800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ca7b0 0x7f693c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f6941800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f693c0ce930 0x7f693c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f6941800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693c159830 0x7f693c14cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f6941800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c15bdb0 con 0x7f693c0ca7b0 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f6941800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c15a7d0 con 0x7f693c159830 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f6941800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693c15aab0 con 0x7f693c0ce930 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f693aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ca7b0 0x7f693c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f693aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ca7b0 0x7f693c15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41436/0 (socket says 172.21.15.105:41436) 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.543+0000 7f693aa00640 1 -- 172.21.15.105:0/935203149 learned_addr learned my addr 172.21.15.105:0/935203149 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:35.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693a000640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f693c0ce930 0x7f693c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693b400640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693c159830 0x7f693c14cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693aa00640 1 -- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f693c0ce930 msgr2=0x7f693c155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693aa00640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f693c0ce930 0x7f693c155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693aa00640 1 -- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693c159830 msgr2=0x7f693c14cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693aa00640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693c159830 0x7f693c14cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693aa00640 1 -- 172.21.15.105:0/935203149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6934009b30 con 0x7f693c0ca7b0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693a000640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f693c0ce930 0x7f693c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f693aa00640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ca7b0 0x7f693c15b870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b3f0 tx=0x7f692c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f6932a00640 1 -- 172.21.15.105:0/935203149 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c011830 con 0x7f693c0ca7b0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f6932a00640 1 -- 172.21.15.105:0/935203149 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c011990 con 0x7f693c0ca7b0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.544+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c15ad90 con 0x7f693c0ca7b0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.545+0000 7f6932a00640 1 -- 172.21.15.105:0/935203149 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c011d40 con 0x7f693c0ca7b0 2024-03-23T07:56:35.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.545+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c15b350 con 0x7f693c0ca7b0 2024-03-23T07:56:35.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.546+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693c15b570 con 0x7f693c0ca7b0 2024-03-23T07:56:35.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.547+0000 7f6932a00640 1 -- 172.21.15.105:0/935203149 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f692c01a400 con 0x7f693c0ca7b0 2024-03-23T07:56:35.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.548+0000 7f6932a00640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691c06ac00 0x7f691c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.548+0000 7f6932a00640 1 -- 172.21.15.105:0/935203149 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f692c089520 con 0x7f693c0ca7b0 2024-03-23T07:56:35.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.548+0000 7f693a000640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691c06ac00 0x7f691c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.548+0000 7f693a000640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691c06ac00 0x7f691c06d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6928006fd0 tx=0x7f6928008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.552+0000 7f6932a00640 1 -- 172.21.15.105:0/935203149 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f692c054380 con 0x7f693c0ca7b0 2024-03-23T07:56:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.745+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f693c073390 con 0x7f693c0ca7b0 2024-03-23T07:56:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.746+0000 7f6932a00640 1 -- 172.21.15.105:0/935203149 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f692c0579a0 con 0x7f693c0ca7b0 2024-03-23T07:56:35.746 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:35.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691c06ac00 msgr2=0x7f691c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691c06ac00 0x7f691c06d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6928006fd0 tx=0x7f6928008040 comp rx=0 tx=0).stop 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ca7b0 msgr2=0x7f693c15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ca7b0 0x7f693c15b870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b3f0 tx=0x7f692c00b8b0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 shutdown_connections 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693c159830 0x7f693c14cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691c06ac00 0x7f691c06d0c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f693c0ce930 0x7f693c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 --2- 172.21.15.105:0/935203149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693c0ca7b0 0x7f693c15b870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 >> 172.21.15.105:0/935203149 conn(0x7f693c06a0b0 msgr2=0x7f693c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 shutdown_connections 2024-03-23T07:56:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.749+0000 7f6941800640 1 -- 172.21.15.105:0/935203149 wait complete. 2024-03-23T07:56:35.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T07:56:35.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.977+0000 7f8e9f800640 1 Processor -- start 2024-03-23T07:56:35.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.977+0000 7f8e9f800640 1 -- start start 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e9f800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e980ce930 0x7f8e980ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e9f800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e9806d2c0 0x7f8e980cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e9f800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e980cf250 con 0x7f8e980cb660 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e9f800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9815bae0 con 0x7f8e980ce930 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e9f800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815ae60 con 0x7f8e9806d2c0 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e9ce00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e980ce930 0x7f8e980ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e8fe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8e980cb660 0x7f8e98070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43668/0 (socket says 172.21.15.105:43668) 2024-03-23T07:56:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.978+0000 7f8e8fe00640 1 -- 172.21.15.105:0/134860894 learned_addr learned my addr 172.21.15.105:0/134860894 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:35.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.979+0000 7f8e9d800640 1 --2- 172.21.15.105:0/134860894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e9806d2c0 0x7f8e980cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.979+0000 7f8e9ce00640 1 -- 172.21.15.105:0/134860894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e9806d2c0 msgr2=0x7f8e980cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:35.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.979+0000 7f8e9ce00640 1 --2- 172.21.15.105:0/134860894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e9806d2c0 0x7f8e980cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.979+0000 7f8e97e00640 1 -- 172.21.15.105:0/134860894 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900477297 0 0) 0x7f8e980cf250 con 0x7f8e980cb660 2024-03-23T07:56:35.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.979+0000 7f8e9ce00640 1 -- 172.21.15.105:0/134860894 >> v1:172.21.15.105:6789/0 conn(0x7f8e980cb660 legacy=0x7f8e98070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.979+0000 7f8e9ce00640 1 -- 172.21.15.105:0/134860894 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9815b140 con 0x7f8e980ce930 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.980+0000 7f8e9ce00640 1 --2- 172.21.15.105:0/134860894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980ce930 0x7f8e980ced10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e930c1b85b3ce884 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.980+0000 7f8e97e00640 1 -- 172.21.15.105:0/134860894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80031830 con 0x7f8e980ce930 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.980+0000 7f8e97e00640 1 -- 172.21.15.105:0/134860894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e80031990 con 0x7f8e980ce930 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.980+0000 7f8e97e00640 1 -- 172.21.15.105:0/134860894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80031d20 con 0x7f8e980ce930 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 -- 172.21.15.105:0/134860894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980ce930 msgr2=0x7f8e980ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 --2- 172.21.15.105:0/134860894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980ce930 0x7f8e980ced10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002b8d0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 -- 172.21.15.105:0/134860894 shutdown_connections 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 --2- 172.21.15.105:0/134860894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e9806d2c0 0x7f8e980cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 --2- 172.21.15.105:0/134860894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980ce930 0x7f8e980ced10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 -- 172.21.15.105:0/134860894 >> 172.21.15.105:0/134860894 conn(0x7f8e9806a0b0 msgr2=0x7f8e9806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 -- 172.21.15.105:0/134860894 shutdown_connections 2024-03-23T07:56:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.981+0000 7f8e9f800640 1 -- 172.21.15.105:0/134860894 wait complete. 2024-03-23T07:56:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.983+0000 7f8e9f800640 1 Processor -- start 2024-03-23T07:56:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.983+0000 7f8e9f800640 1 -- start start 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9f800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e9806d2c0 0x7f8e981f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9f800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e980cb660 0x7f8e9813e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9f800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e980ce930 0x7f8e98159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9f800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e981f6230 con 0x7f8e9806d2c0 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9f800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9813e9e0 con 0x7f8e980ce930 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9f800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9813da10 con 0x7f8e980cb660 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e9806d2c0 0x7f8e981f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e9806d2c0 0x7f8e981f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41460/0 (socket says 172.21.15.105:41460) 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e9ce00640 1 -- 172.21.15.105:0/292938788 learned_addr learned my addr 172.21.15.105:0/292938788 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.984+0000 7f8e8fe00640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e980cb660 0x7f8e9813e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9d800640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e980ce930 0x7f8e98159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9ce00640 1 -- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e980cb660 msgr2=0x7f8e9813e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9ce00640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e980cb660 0x7f8e9813e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9ce00640 1 -- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e980ce930 msgr2=0x7f8e98159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9ce00640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e980ce930 0x7f8e98159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9ce00640 1 -- 172.21.15.105:0/292938788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80009400 con 0x7f8e9806d2c0 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9ce00640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e9806d2c0 0x7f8e981f5cf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e8002bee0 tx=0x7f8e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e95600640 1 -- 172.21.15.105:0/292938788 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8003a6f0 con 0x7f8e9806d2c0 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e95600640 1 -- 172.21.15.105:0/292938788 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8003a870 con 0x7f8e9806d2c0 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e98155280 con 0x7f8e9806d2c0 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.985+0000 7f8e95600640 1 -- 172.21.15.105:0/292938788 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8003ac20 con 0x7f8e9806d2c0 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.986+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e981580b0 con 0x7f8e9806d2c0 2024-03-23T07:56:35.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.987+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e98158240 con 0x7f8e9806d2c0 2024-03-23T07:56:35.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.988+0000 7f8e95600640 1 -- 172.21.15.105:0/292938788 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8e8004c3b0 con 0x7f8e9806d2c0 2024-03-23T07:56:35.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.989+0000 7f8e95600640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e7406ac00 0x7f8e7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:35.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.989+0000 7f8e95600640 1 -- 172.21.15.105:0/292938788 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8e800ab510 con 0x7f8e9806d2c0 2024-03-23T07:56:35.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.989+0000 7f8e8fe00640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e7406ac00 0x7f8e7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:35.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.990+0000 7f8e8fe00640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e7406ac00 0x7f8e7406d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e980cbdd0 tx=0x7f8e88007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:35.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:35.993+0000 7f8e95600640 1 -- 172.21.15.105:0/292938788 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8e80076480 con 0x7f8e9806d2c0 2024-03-23T07:56:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.200+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8e98073390 con 0x7f8e9806d2c0 2024-03-23T07:56:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.201+0000 7f8e95600640 1 -- 172.21.15.105:0/292938788 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8e80079aa0 con 0x7f8e9806d2c0 2024-03-23T07:56:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e7406ac00 msgr2=0x7f8e7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e7406ac00 0x7f8e7406d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e980cbdd0 tx=0x7f8e88007b50 comp rx=0 tx=0).stop 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e9806d2c0 msgr2=0x7f8e981f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e9806d2c0 0x7f8e981f5cf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e8002bee0 tx=0x7f8e800046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 shutdown_connections 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e980ce930 0x7f8e98159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e7406ac00 0x7f8e7406d0c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e980cb660 0x7f8e9813e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 --2- 172.21.15.105:0/292938788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e9806d2c0 0x7f8e981f5cf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 >> 172.21.15.105:0/292938788 conn(0x7f8e9806a0b0 msgr2=0x7f8e9806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 shutdown_connections 2024-03-23T07:56:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:36.204+0000 7f8e9f800640 1 -- 172.21.15.105:0/292938788 wait complete. 2024-03-23T07:56:36.221 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-03-23T07:56:36.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:36.221 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-23T07:56:36.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[43624]: fuse finished with error 0 and tester_r 0 2024-03-23T07:56:36.311 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:56:37.866 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:56:37.874 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:56:42.412 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:56:42.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:42.412 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-23T07:56:42.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.701+0000 7fade8a00640 1 Processor -- start 2024-03-23T07:56:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.701+0000 7fade8a00640 1 -- start start 2024-03-23T07:56:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.702+0000 7fade8a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fade4156e30 0x7fade406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.702+0000 7fade8a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fade40cb540 0x7fade40b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.702+0000 7fade8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40cc8c0 con 0x7fade40b3310 2024-03-23T07:56:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.702+0000 7fade8a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade406d630 con 0x7fade40cb540 2024-03-23T07:56:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.702+0000 7fade8a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade415ce60 con 0x7fade4156e30 2024-03-23T07:56:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fade40cb540 0x7fade40b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:42.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fade40cb540 0x7fade40b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46450/0 (socket says 172.21.15.105:46450) 2024-03-23T07:56:42.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3e00640 1 -- 172.21.15.105:0/4018164400 learned_addr learned my addr 172.21.15.105:0/4018164400 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:42.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3400640 1 --2- 172.21.15.105:0/4018164400 >> v2:172.21.15.132:3300/0 conn(0x7fade4156e30 0x7fade406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:42.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3e00640 1 -- 172.21.15.105:0/4018164400 >> v2:172.21.15.132:3300/0 conn(0x7fade4156e30 msgr2=0x7fade406d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:42.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3e00640 1 --2- 172.21.15.105:0/4018164400 >> v2:172.21.15.132:3300/0 conn(0x7fade4156e30 0x7fade406d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3e00640 1 -- 172.21.15.105:0/4018164400 >> v1:172.21.15.132:6789/0 conn(0x7fade40b3310 legacy=0x7fade40cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:42.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.703+0000 7fade3e00640 1 -- 172.21.15.105:0/4018164400 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade415d120 con 0x7fade40cb540 2024-03-23T07:56:42.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.704+0000 7fade3e00640 1 --2- 172.21.15.105:0/4018164400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40cb540 0x7fade40b1d70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fade406ace0 tx=0x7faddc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1139279839c88d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade2a00640 1 -- 172.21.15.105:0/4018164400 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc02ca80 con 0x7fade40cb540 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade2a00640 1 -- 172.21.15.105:0/4018164400 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc02cbe0 con 0x7fade40cb540 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade2a00640 1 -- 172.21.15.105:0/4018164400 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc0343f0 con 0x7fade40cb540 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade8a00640 1 -- 172.21.15.105:0/4018164400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40cb540 msgr2=0x7fade40b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade8a00640 1 --2- 172.21.15.105:0/4018164400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40cb540 0x7fade40b1d70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fade406ace0 tx=0x7faddc02bee0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade8a00640 1 -- 172.21.15.105:0/4018164400 shutdown_connections 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade8a00640 1 --2- 172.21.15.105:0/4018164400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40cb540 0x7fade40b1d70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade8a00640 1 --2- 172.21.15.105:0/4018164400 >> v2:172.21.15.132:3300/0 conn(0x7fade4156e30 0x7fade406d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.705+0000 7fade8a00640 1 -- 172.21.15.105:0/4018164400 >> 172.21.15.105:0/4018164400 conn(0x7fade4065df0 msgr2=0x7fade4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:42.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.706+0000 7fade8a00640 1 -- 172.21.15.105:0/4018164400 shutdown_connections 2024-03-23T07:56:42.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.706+0000 7fade8a00640 1 -- 172.21.15.105:0/4018164400 wait complete. 2024-03-23T07:56:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.708+0000 7fade8a00640 1 Processor -- start 2024-03-23T07:56:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.708+0000 7fade8a00640 1 -- start start 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40b3310 0x7fade4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade40cb540 0x7fade41430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fade4156e30 0x7fade4150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade8a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade414dd00 con 0x7fade40b3310 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41440f0 con 0x7fade40cb540 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade414f1c0 con 0x7fade4156e30 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40b3310 0x7fade4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40b3310 0x7fade4153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46466/0 (socket says 172.21.15.105:46466) 2024-03-23T07:56:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade3400640 1 -- 172.21.15.105:0/1681380548 learned_addr learned my addr 172.21.15.105:0/1681380548 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.709+0000 7fade3e00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fade4156e30 0x7fade4150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7faddaa00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade40cb540 0x7fade41430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade3400640 1 -- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fade4156e30 msgr2=0x7fade4150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade3400640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fade4156e30 0x7fade4150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade3400640 1 -- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade40cb540 msgr2=0x7fade41430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade3400640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade40cb540 0x7fade41430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade3400640 1 -- 172.21.15.105:0/1681380548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc009b30 con 0x7fade40b3310 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7faddaa00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade40cb540 0x7fade41430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade3400640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40b3310 0x7fade4153c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fadd400d800 tx=0x7fadd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7faddbe00640 1 -- 172.21.15.105:0/1681380548 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400b6c0 con 0x7fade40b3310 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7faddbe00640 1 -- 172.21.15.105:0/1681380548 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd400b820 con 0x7fade40b3310 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade414f460 con 0x7fade40b3310 2024-03-23T07:56:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7faddbe00640 1 -- 172.21.15.105:0/1681380548 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400bbb0 con 0x7fade40b3310 2024-03-23T07:56:42.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.710+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade414fa20 con 0x7fade40b3310 2024-03-23T07:56:42.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.711+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade40cd880 con 0x7fade40b3310 2024-03-23T07:56:42.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.712+0000 7faddbe00640 1 -- 172.21.15.105:0/1681380548 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fadd400bd10 con 0x7fade40b3310 2024-03-23T07:56:42.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.713+0000 7faddbe00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadc006ac00 0x7fadc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.713+0000 7faddbe00640 1 -- 172.21.15.105:0/1681380548 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fadd4089c70 con 0x7fade40b3310 2024-03-23T07:56:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.714+0000 7faddaa00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadc006ac00 0x7fadc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.714+0000 7faddaa00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadc006ac00 0x7fadc006d0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fade4153400 tx=0x7fadcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:42.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.718+0000 7faddbe00640 1 -- 172.21.15.105:0/1681380548 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fadd4054ac0 con 0x7fade40b3310 2024-03-23T07:56:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.913+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fade404e030 con 0x7fade40b3310 2024-03-23T07:56:42.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.916+0000 7faddbe00640 1 -- 172.21.15.105:0/1681380548 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7fadd40580e0 con 0x7fade40b3310 2024-03-23T07:56:42.917 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:42.917 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadc006ac00 msgr2=0x7fadc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadc006ac00 0x7fadc006d0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fade4153400 tx=0x7fadcc008040 comp rx=0 tx=0).stop 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40b3310 msgr2=0x7fade4153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40b3310 0x7fade4153c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fadd400d800 tx=0x7fadd400dcc0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 shutdown_connections 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fadc006ac00 0x7fadc006d0c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fade4156e30 0x7fade4150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade40cb540 0x7fade41430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 --2- 172.21.15.105:0/1681380548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fade40b3310 0x7fade4153c40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 >> 172.21.15.105:0/1681380548 conn(0x7fade4065df0 msgr2=0x7fade4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 shutdown_connections 2024-03-23T07:56:42.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:42.919+0000 7fade8a00640 1 -- 172.21.15.105:0/1681380548 wait complete. 2024-03-23T07:56:42.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:43.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.147+0000 7f357d800640 1 Processor -- start 2024-03-23T07:56:43.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.147+0000 7f357d800640 1 -- start start 2024-03-23T07:56:43.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.149+0000 7f357d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 0x7f35780b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.149+0000 7f357d800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35780d1830 con 0x7f357815cda0 2024-03-23T07:56:43.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.149+0000 7f357d800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35780b1f20 con 0x7f35780b5a30 2024-03-23T07:56:43.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.149+0000 7f357d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35780d0c20 con 0x7f35780b4a60 2024-03-23T07:56:43.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.149+0000 7f3576a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f357815cda0 0x7f35780cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42984/0 (socket says 172.21.15.105:42984) 2024-03-23T07:56:43.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.149+0000 7f3576a00640 1 -- 172.21.15.105:0/777431359 learned_addr learned my addr 172.21.15.105:0/777431359 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:43.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3577400640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 0x7f35780b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018154250 0 0) 0x7f35780d1830 con 0x7f357815cda0 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f355c003520 con 0x7f357815cda0 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518665570 0 0) 0x7f355c003520 con 0x7f357815cda0 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 msgr2=0x7f35780b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 0x7f35780b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 >> v1:172.21.15.132:6789/0 conn(0x7f35780b5a30 legacy=0x7f35780b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780d0f00 con 0x7f357815cda0 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f35700019f0 con 0x7f357815cda0 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 >> v1:172.21.15.105:6789/0 conn(0x7f357815cda0 legacy=0x7f35780cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 0x7f355c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c003f70 0x7f355c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c004d00 0x7f355c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780d0c20 con 0x7f355c004d00 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c00bac0 con 0x7f355c003f70 2024-03-23T07:56:43.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c00bc80 con 0x7f35780b4a60 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.150+0000 7f3577400640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c004d00 0x7f355c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.151+0000 7f3576a00640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 0x7f355c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.151+0000 7f3576000640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c003f70 0x7f355c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.151+0000 7f3577400640 1 -- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 msgr2=0x7f355c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.151+0000 7f3577400640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35780b4a60 0x7f355c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.151+0000 7f3577400640 1 -- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c003f70 msgr2=0x7f355c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.151+0000 7f3577400640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c003f70 0x7f355c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.151+0000 7f3577400640 1 -- 172.21.15.105:0/777431359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c003520 con 0x7f355c004d00 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.152+0000 7f3577400640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c004d00 0x7f355c005180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3568009a90 tx=0x7f355c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c42dfd3a3dff6fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.152+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356802c680 con 0x7f355c004d00 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.152+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356802c7e0 con 0x7f355c004d00 2024-03-23T07:56:43.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.152+0000 7f3575600640 1 -- 172.21.15.105:0/777431359 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356802cb70 con 0x7f355c004d00 2024-03-23T07:56:43.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.153+0000 7f357d800640 1 -- 172.21.15.105:0/777431359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c004d00 msgr2=0x7f355c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.153+0000 7f357d800640 1 --2- 172.21.15.105:0/777431359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c004d00 0x7f355c005180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3568009a90 tx=0x7f355c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.153+0000 7f3577400640 1 -- 172.21.15.105:0/777431359 reap_dead start 2024-03-23T07:56:43.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.153+0000 7f357d800640 1 -- 172.21.15.105:0/777431359 shutdown_connections 2024-03-23T07:56:43.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.153+0000 7f357d800640 1 -- 172.21.15.105:0/777431359 >> 172.21.15.105:0/777431359 conn(0x7f35780cf350 msgr2=0x7f35780cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:43.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.153+0000 7f357d800640 1 -- 172.21.15.105:0/777431359 shutdown_connections 2024-03-23T07:56:43.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.153+0000 7f357d800640 1 -- 172.21.15.105:0/777431359 wait complete. 2024-03-23T07:56:43.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.155+0000 7f357d800640 1 Processor -- start 2024-03-23T07:56:43.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.155+0000 7f357d800640 1 -- start start 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f357d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c004d00 0x7f35780b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f357d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35781f2bf0 0x7f3578158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f357d800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35780c8a00 0x7f35780c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f357d800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3578159410 con 0x7f35780c8a00 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f357d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780ca9f0 con 0x7f355c004d00 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f357d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35781f3070 con 0x7f35781f2bf0 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35780c8a00 0x7f35780c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35780c8a00 0x7f35780c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46486/0 (socket says 172.21.15.105:46486) 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 -- 172.21.15.105:0/1554114460 learned_addr learned my addr 172.21.15.105:0/1554114460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 -- 172.21.15.105:0/1554114460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35781f2bf0 msgr2=0x7f3578158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35781f2bf0 0x7f3578158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 -- 172.21.15.105:0/1554114460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c004d00 msgr2=0x7f35780b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3576a00640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c004d00 0x7f35780b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c004d00 0x7f35780b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.156+0000 7f3577400640 1 -- 172.21.15.105:0/1554114460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780c6dd0 con 0x7f35780c8a00 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.157+0000 7f3577400640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35780c8a00 0x7f35780c8e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3568009a90 tx=0x7f3568004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.157+0000 7f356ea00640 1 -- 172.21.15.105:0/1554114460 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356802c680 con 0x7f35780c8a00 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.157+0000 7f356ea00640 1 -- 172.21.15.105:0/1554114460 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3568003830 con 0x7f35780c8a00 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.157+0000 7f356ea00640 1 -- 172.21.15.105:0/1554114460 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356802dea0 con 0x7f35780c8a00 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.157+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35780c7090 con 0x7f35780c8a00 2024-03-23T07:56:43.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.157+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35780cac40 con 0x7f35780c8a00 2024-03-23T07:56:43.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.158+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35780c75b0 con 0x7f35780c8a00 2024-03-23T07:56:43.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.159+0000 7f356ea00640 1 -- 172.21.15.105:0/1554114460 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3568003be0 con 0x7f35780c8a00 2024-03-23T07:56:43.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.160+0000 7f356ea00640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f355406ac00 0x7f355406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.161+0000 7f356ea00640 1 -- 172.21.15.105:0/1554114460 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f35680aae30 con 0x7f35780c8a00 2024-03-23T07:56:43.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.161+0000 7f3576a00640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f355406ac00 0x7f355406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.161+0000 7f3576a00640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f355406ac00 0x7f355406d0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3570002230 tx=0x7f35700012d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:43.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.165+0000 7f356ea00640 1 -- 172.21.15.105:0/1554114460 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3568075d10 con 0x7f35780c8a00 2024-03-23T07:56:43.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.359+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35780cae30 con 0x7f35780c8a00 2024-03-23T07:56:43.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.362+0000 7f356ea00640 1 -- 172.21.15.105:0/1554114460 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7f3568042090 con 0x7f35780c8a00 2024-03-23T07:56:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:43.364 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:43.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f355406ac00 msgr2=0x7f355406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f355406ac00 0x7f355406d0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3570002230 tx=0x7f35700012d0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35780c8a00 msgr2=0x7f35780c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35780c8a00 0x7f35780c8e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3568009a90 tx=0x7f3568004730 comp rx=0 tx=0).stop 2024-03-23T07:56:43.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 shutdown_connections 2024-03-23T07:56:43.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35780c8a00 0x7f35780c8e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f355406ac00 0x7f355406d0c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35781f2bf0 0x7f3578158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 --2- 172.21.15.105:0/1554114460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f355c004d00 0x7f35780b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 >> 172.21.15.105:0/1554114460 conn(0x7f35780cf350 msgr2=0x7f35780b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:43.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 shutdown_connections 2024-03-23T07:56:43.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.365+0000 7f357d800640 1 -- 172.21.15.105:0/1554114460 wait complete. 2024-03-23T07:56:43.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T07:56:43.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.594+0000 7f9db7c00640 1 Processor -- start 2024-03-23T07:56:43.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.594+0000 7f9db7c00640 1 -- start start 2024-03-23T07:56:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db7c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9db0069c90 0x7f9db006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9db00b26e0 0x7f9db00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db7c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db006fc80 con 0x7f9db015ca00 2024-03-23T07:56:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db7c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db006a660 con 0x7f9db0069c90 2024-03-23T07:56:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00cf310 con 0x7f9db00b26e0 2024-03-23T07:56:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db5200640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9db015ca00 0x7f9db006f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42998/0 (socket says 172.21.15.105:42998) 2024-03-23T07:56:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db5200640 1 -- 172.21.15.105:0/1650804590 learned_addr learned my addr 172.21.15.105:0/1650804590 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267472880 0 0) 0x7f9db006fc80 con 0x7f9db015ca00 2024-03-23T07:56:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d94003520 con 0x7f9db015ca00 2024-03-23T07:56:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.596+0000 7f9db5c00640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9db00b26e0 0x7f9db00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568599350 0 0) 0x7f9d94003520 con 0x7f9db015ca00 2024-03-23T07:56:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9db00b26e0 msgr2=0x7f9db00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9db00b26e0 0x7f9db00b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 >> v2:172.21.15.132:3300/0 conn(0x7f9db0069c90 msgr2=0x7f9db006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 --2- 172.21.15.105:0/1650804590 >> v2:172.21.15.132:3300/0 conn(0x7f9db0069c90 0x7f9db006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00cf5f0 con 0x7f9db015ca00 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f9da0001670 con 0x7f9db015ca00 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 >> v1:172.21.15.105:6789/0 conn(0x7f9db015ca00 legacy=0x7f9db006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9db00b26e0 0x7f9d94003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d94003fa0 0x7f9d940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d94004d00 0x7f9d94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db006a660 con 0x7f9d94004d00 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00cf310 con 0x7f9d94003fa0 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9400bb90 con 0x7f9db00b26e0 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.597+0000 7f9db5c00640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d94004d00 0x7f9d94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.598+0000 7f9db5c00640 1 -- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9db00b26e0 msgr2=0x7f9d94003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.598+0000 7f9db5c00640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9db00b26e0 0x7f9d94003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.598+0000 7f9db5c00640 1 -- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d94003fa0 msgr2=0x7f9d940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.598+0000 7f9db5c00640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d94003fa0 0x7f9d940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.598+0000 7f9db5c00640 1 -- 172.21.15.105:0/1650804590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94003520 con 0x7f9d94004d00 2024-03-23T07:56:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9db5c00640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d94004d00 0x7f9d94005180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9da8007360 tx=0x7f9d940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36ca7485704e7a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da802ccf0 con 0x7f9d94004d00 2024-03-23T07:56:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da802ce50 con 0x7f9d94004d00 2024-03-23T07:56:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9daf400640 1 -- 172.21.15.105:0/1650804590 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8034660 con 0x7f9d94004d00 2024-03-23T07:56:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9db7c00640 1 -- 172.21.15.105:0/1650804590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d94004d00 msgr2=0x7f9d94005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9db7c00640 1 --2- 172.21.15.105:0/1650804590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d94004d00 0x7f9d94005180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9da8007360 tx=0x7f9d940057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9db5c00640 1 -- 172.21.15.105:0/1650804590 reap_dead start 2024-03-23T07:56:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9db7c00640 1 -- 172.21.15.105:0/1650804590 shutdown_connections 2024-03-23T07:56:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.599+0000 7f9db7c00640 1 -- 172.21.15.105:0/1650804590 >> 172.21.15.105:0/1650804590 conn(0x7f9db006d440 msgr2=0x7f9db006d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.600+0000 7f9db7c00640 1 -- 172.21.15.105:0/1650804590 shutdown_connections 2024-03-23T07:56:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.600+0000 7f9db7c00640 1 -- 172.21.15.105:0/1650804590 wait complete. 2024-03-23T07:56:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.601+0000 7f9db7c00640 1 Processor -- start 2024-03-23T07:56:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.601+0000 7f9db7c00640 1 -- start start 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.602+0000 7f9db7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d94004d00 0x7f9db0157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.602+0000 7f9db7c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db00c8a60 0x7f9db0144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.602+0000 7f9db7c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9db01f5700 0x7f9db01f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.602+0000 7f9db7c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00ca6b0 con 0x7f9db00c8a60 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.602+0000 7f9db7c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db01f60c0 con 0x7f9db01f5700 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.602+0000 7f9db7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0144b60 con 0x7f9d94004d00 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db00c8a60 0x7f9db0144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db00c8a60 0x7f9db0144620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46508/0 (socket says 172.21.15.105:46508) 2024-03-23T07:56:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 -- 172.21.15.105:0/131402511 learned_addr learned my addr 172.21.15.105:0/131402511 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9db5c00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9db01f5700 0x7f9db01f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9db5200640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d94004d00 0x7f9db0157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 -- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d94004d00 msgr2=0x7f9db0157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d94004d00 0x7f9db0157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 -- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9db01f5700 msgr2=0x7f9db01f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9db01f5700 0x7f9db01f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 -- 172.21.15.105:0/131402511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0157b40 con 0x7f9db00c8a60 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.603+0000 7f9dafe00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db00c8a60 0x7f9db0144620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009fd0 tx=0x7f9d9c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.604+0000 7f9dacc00640 1 -- 172.21.15.105:0/131402511 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c011830 con 0x7f9db00c8a60 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.604+0000 7f9dacc00640 1 -- 172.21.15.105:0/131402511 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c011990 con 0x7f9db00c8a60 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.604+0000 7f9dacc00640 1 -- 172.21.15.105:0/131402511 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c011d20 con 0x7f9db00c8a60 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.604+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db0157e20 con 0x7f9db00c8a60 2024-03-23T07:56:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.604+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db01583e0 con 0x7f9db00c8a60 2024-03-23T07:56:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.605+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db00ca7f0 con 0x7f9db00c8a60 2024-03-23T07:56:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.606+0000 7f9dacc00640 1 -- 172.21.15.105:0/131402511 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9d9c01b460 con 0x7f9db00c8a60 2024-03-23T07:56:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.607+0000 7f9dacc00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9d9006ac00 0x7f9d9006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.607+0000 7f9dacc00640 1 -- 172.21.15.105:0/131402511 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9d9c0893e0 con 0x7f9db00c8a60 2024-03-23T07:56:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.607+0000 7f9db5200640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9d9006ac00 0x7f9d9006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:43.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.608+0000 7f9db5200640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9d9006ac00 0x7f9d9006d0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9da0007ab0 tx=0x7f9da0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:43.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.611+0000 7f9dacc00640 1 -- 172.21.15.105:0/131402511 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9db00ca7f0 con 0x7f9db00c8a60 2024-03-23T07:56:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:43.826+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9db00ca7f0 con 0x7f9db00c8a60 2024-03-23T07:56:44.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.125+0000 7f9dacc00640 1 -- 172.21.15.105:0/131402511 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9db00ca7f0 con 0x7f9db00c8a60 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.128+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9d9006ac00 msgr2=0x7f9d9006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.128+0000 7f9db7c00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9d9006ac00 0x7f9d9006d0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9da0007ab0 tx=0x7f9da0009040 comp rx=0 tx=0).stop 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.128+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db00c8a60 msgr2=0x7f9db0144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.128+0000 7f9db7c00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db00c8a60 0x7f9db0144620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009fd0 tx=0x7f9d9c00b8e0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 shutdown_connections 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9db01f5700 0x7f9db01f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db00c8a60 0x7f9db0144620 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9d9006ac00 0x7f9d9006d0c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 --2- 172.21.15.105:0/131402511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d94004d00 0x7f9db0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 >> 172.21.15.105:0/131402511 conn(0x7f9db006d440 msgr2=0x7f9db00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 shutdown_connections 2024-03-23T07:56:44.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.129+0000 7f9db7c00640 1 -- 172.21.15.105:0/131402511 wait complete. 2024-03-23T07:56:44.150 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-23T07:56:44.151 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T07:56:44.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.359+0000 7fa546c00640 1 Processor -- start 2024-03-23T07:56:44.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.359+0000 7fa546c00640 1 -- start start 2024-03-23T07:56:44.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.360+0000 7fa546c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5400cd740 0x7fa5400b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.360+0000 7fa546c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400d20c0 con 0x7fa5400d0d40 2024-03-23T07:56:44.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.360+0000 7fa546c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400d14b0 con 0x7fa5400b5aa0 2024-03-23T07:56:44.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.360+0000 7fa546c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400d1790 con 0x7fa5400cd740 2024-03-23T07:56:44.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.361+0000 7fa53f400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa5400d0d40 0x7fa5400cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43012/0 (socket says 172.21.15.105:43012) 2024-03-23T07:56:44.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.361+0000 7fa53f400640 1 -- 172.21.15.105:0/3537015652 learned_addr learned my addr 172.21.15.105:0/3537015652 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:44.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.361+0000 7fa53fe00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5400cd740 0x7fa5400b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.361+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132236692 0 0) 0x7fa5400d20c0 con 0x7fa5400d0d40 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.361+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa524003520 con 0x7fa5400d0d40 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112085719 0 0) 0x7fa524003520 con 0x7fa5400d0d40 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5400cd740 msgr2=0x7fa5400b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5400cd740 0x7fa5400b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 >> v1:172.21.15.132:6789/0 conn(0x7fa5400b5aa0 legacy=0x7fa5400b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400d1a70 con 0x7fa5400d0d40 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fa52c001670 con 0x7fa5400d0d40 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 >> v1:172.21.15.105:6789/0 conn(0x7fa5400d0d40 legacy=0x7fa5400cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5240039a0 0x7fa524003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5240047c0 0x7fa524004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa524005180 0x7fa52400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400d1790 con 0x7fa5240039a0 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52400c050 con 0x7fa524005180 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52400c210 con 0x7fa5240047c0 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53fe00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5400cd740 0x7fa5400b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53fe00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5240039a0 0x7fa524003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa544c00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa524005180 0x7fa52400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.362+0000 7fa53f400640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5240047c0 0x7fa524004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.363+0000 7fa53fe00640 1 -- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5240047c0 msgr2=0x7fa524004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.363+0000 7fa53fe00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5240047c0 0x7fa524004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.363+0000 7fa53fe00640 1 -- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa524005180 msgr2=0x7fa52400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.363+0000 7fa53fe00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa524005180 0x7fa52400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.363+0000 7fa53fe00640 1 -- 172.21.15.105:0/3537015652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa524003520 con 0x7fa5240039a0 2024-03-23T07:56:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.363+0000 7fa544c00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa524005180 0x7fa52400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa53fe00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5240039a0 0x7fa524003dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa5240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eb5a0ce31d6c71 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530033600 con 0x7fa5240039a0 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53002f050 con 0x7fa5240039a0 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa53ea00640 1 -- 172.21.15.105:0/3537015652 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530033980 con 0x7fa5240039a0 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa546c00640 1 -- 172.21.15.105:0/3537015652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5240039a0 msgr2=0x7fa524003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa546c00640 1 --2- 172.21.15.105:0/3537015652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5240039a0 0x7fa524003dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa5240057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa544c00640 1 -- 172.21.15.105:0/3537015652 reap_dead start 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa546c00640 1 -- 172.21.15.105:0/3537015652 shutdown_connections 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.364+0000 7fa546c00640 1 -- 172.21.15.105:0/3537015652 >> 172.21.15.105:0/3537015652 conn(0x7fa5400cf5d0 msgr2=0x7fa5400cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:44.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.365+0000 7fa546c00640 1 -- 172.21.15.105:0/3537015652 shutdown_connections 2024-03-23T07:56:44.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.365+0000 7fa546c00640 1 -- 172.21.15.105:0/3537015652 wait complete. 2024-03-23T07:56:44.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.366+0000 7fa546c00640 1 Processor -- start 2024-03-23T07:56:44.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.366+0000 7fa546c00640 1 -- start start 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa546c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5240039a0 0x7fa540153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa546c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa540154840 0x7fa5401552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa546c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5401542f0 0x7fa5401557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa546c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400cc120 con 0x7fa5401542f0 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa546c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400b11e0 con 0x7fa5240039a0 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa546c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400cb550 con 0x7fa540154840 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa544c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5401542f0 0x7fa5401557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa544c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5401542f0 0x7fa5401557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46532/0 (socket says 172.21.15.105:46532) 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.367+0000 7fa544c00640 1 -- 172.21.15.105:0/2450066647 learned_addr learned my addr 172.21.15.105:0/2450066647 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa53fe00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5240039a0 0x7fa540153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa53f400640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa540154840 0x7fa5401552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa544c00640 1 -- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa540154840 msgr2=0x7fa5401552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa544c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa540154840 0x7fa5401552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa544c00640 1 -- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5240039a0 msgr2=0x7fa540153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa544c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5240039a0 0x7fa540153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa544c00640 1 -- 172.21.15.105:0/2450066647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400cb7f0 con 0x7fa5401542f0 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa544c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5401542f0 0x7fa5401557e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa538000f80 tx=0x7fa53800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa537e00640 1 -- 172.21.15.105:0/2450066647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538017070 con 0x7fa5401542f0 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.368+0000 7fa537e00640 1 -- 172.21.15.105:0/2450066647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538006b50 con 0x7fa5401542f0 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.369+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5400cbad0 con 0x7fa5401542f0 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.369+0000 7fa537e00640 1 -- 172.21.15.105:0/2450066647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538006d00 con 0x7fa5401542f0 2024-03-23T07:56:44.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.369+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa540154080 con 0x7fa5401542f0 2024-03-23T07:56:44.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.370+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5400cc290 con 0x7fa5401542f0 2024-03-23T07:56:44.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.371+0000 7fa537e00640 1 -- 172.21.15.105:0/2450066647 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa538003970 con 0x7fa5401542f0 2024-03-23T07:56:44.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.371+0000 7fa537e00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa52006ac00 0x7fa52006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:44.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.372+0000 7fa537e00640 1 -- 172.21.15.105:0/2450066647 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa538088ee0 con 0x7fa5401542f0 2024-03-23T07:56:44.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.372+0000 7fa53fe00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa52006ac00 0x7fa52006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.372+0000 7fa53fe00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa52006ac00 0x7fa52006d0c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa530031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:44.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.376+0000 7fa537e00640 1 -- 172.21.15.105:0/2450066647 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa538053ca0 con 0x7fa5401542f0 2024-03-23T07:56:44.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:44.595+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa540073390 con 0x7fa5401542f0 2024-03-23T07:56:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.129+0000 7fa537e00640 1 -- 172.21.15.105:0/2450066647 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa5380572c0 con 0x7fa5401542f0 2024-03-23T07:56:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa52006ac00 msgr2=0x7fa52006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa52006ac00 0x7fa52006d0c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa530031000 comp rx=0 tx=0).stop 2024-03-23T07:56:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5401542f0 msgr2=0x7fa5401557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5401542f0 0x7fa5401557e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa538000f80 tx=0x7fa53800eb90 comp rx=0 tx=0).stop 2024-03-23T07:56:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 shutdown_connections 2024-03-23T07:56:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5401542f0 0x7fa5401557e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa52006ac00 0x7fa52006d0c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa540154840 0x7fa5401552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 --2- 172.21.15.105:0/2450066647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5240039a0 0x7fa540153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 >> 172.21.15.105:0/2450066647 conn(0x7fa5400cf5d0 msgr2=0x7fa5400b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.133+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 shutdown_connections 2024-03-23T07:56:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.134+0000 7fa546c00640 1 -- 172.21.15.105:0/2450066647 wait complete. 2024-03-23T07:56:45.155 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T07:56:45.155 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:56:45.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.364+0000 7f7390400640 1 Processor -- start 2024-03-23T07:56:45.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.364+0000 7f7390400640 1 -- start start 2024-03-23T07:56:45.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.365+0000 7f7390400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f738813faa0 0x7f7388160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.365+0000 7f7390400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f73880b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f7390400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73880b2030 con 0x7f73880b1be0 2024-03-23T07:56:45.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f7390400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73880b38a0 con 0x7f738813faa0 2024-03-23T07:56:45.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f7390400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880b3b80 con 0x7f7388160800 2024-03-23T07:56:45.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f738da00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f738813faa0 0x7f7388160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f738da00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f738813faa0 0x7f7388160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46534/0 (socket says 172.21.15.105:46534) 2024-03-23T07:56:45.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f738da00640 1 -- 172.21.15.105:0/106266680 learned_addr learned my addr 172.21.15.105:0/106266680 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:45.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f738d000640 1 --2- 172.21.15.105:0/106266680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f73880b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f738da00640 1 -- 172.21.15.105:0/106266680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 msgr2=0x7f73880b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:45.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f738da00640 1 --2- 172.21.15.105:0/106266680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f73880b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.366+0000 7f738da00640 1 -- 172.21.15.105:0/106266680 >> v1:172.21.15.132:6789/0 conn(0x7f73880b1be0 legacy=0x7f73881422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:45.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.367+0000 7f738da00640 1 -- 172.21.15.105:0/106266680 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880b3e60 con 0x7f738813faa0 2024-03-23T07:56:45.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.368+0000 7f738da00640 1 --2- 172.21.15.105:0/106266680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738813faa0 0x7f7388160f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987f6ffacb263786 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:45.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.368+0000 7f7387e00640 1 -- 172.21.15.105:0/106266680 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c031640 con 0x7f738813faa0 2024-03-23T07:56:45.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.368+0000 7f7387e00640 1 -- 172.21.15.105:0/106266680 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c0317a0 con 0x7f738813faa0 2024-03-23T07:56:45.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.368+0000 7f7387e00640 1 -- 172.21.15.105:0/106266680 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c031b30 con 0x7f738813faa0 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 -- 172.21.15.105:0/106266680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738813faa0 msgr2=0x7f7388160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 --2- 172.21.15.105:0/106266680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738813faa0 0x7f7388160f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c02b8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 -- 172.21.15.105:0/106266680 shutdown_connections 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 --2- 172.21.15.105:0/106266680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f73880b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 --2- 172.21.15.105:0/106266680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738813faa0 0x7f7388160f30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 -- 172.21.15.105:0/106266680 >> 172.21.15.105:0/106266680 conn(0x7f738806a3d0 msgr2=0x7f73880b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 -- 172.21.15.105:0/106266680 shutdown_connections 2024-03-23T07:56:45.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.369+0000 7f7390400640 1 -- 172.21.15.105:0/106266680 wait complete. 2024-03-23T07:56:45.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.371+0000 7f7390400640 1 Processor -- start 2024-03-23T07:56:45.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.371+0000 7f7390400640 1 -- start start 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f7390400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73880b1be0 0x7f73880ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f7390400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738813faa0 0x7f73881f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f7390400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f7388142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f7390400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881419b0 con 0x7f73880b1be0 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f7390400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7388141c90 con 0x7f738813faa0 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f7390400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7388141f70 con 0x7f7388160800 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f738da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73880b1be0 0x7f73880ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f738da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73880b1be0 0x7f73880ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46546/0 (socket says 172.21.15.105:46546) 2024-03-23T07:56:45.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f738da00640 1 -- 172.21.15.105:0/2629360028 learned_addr learned my addr 172.21.15.105:0/2629360028 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f738e400640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f7388142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.372+0000 7f738d000640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738813faa0 0x7f73881f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f738da00640 1 -- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 msgr2=0x7f7388142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f738da00640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f7388142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f738da00640 1 -- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738813faa0 msgr2=0x7f73881f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f738da00640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738813faa0 0x7f73881f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f738da00640 1 -- 172.21.15.105:0/2629360028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c0094d0 con 0x7f73880b1be0 2024-03-23T07:56:45.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f738da00640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73880b1be0 0x7f73880ccfc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:45.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f7385600640 1 -- 172.21.15.105:0/2629360028 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c03ac60 con 0x7f73880b1be0 2024-03-23T07:56:45.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f7385600640 1 -- 172.21.15.105:0/2629360028 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c03adc0 con 0x7f73880b1be0 2024-03-23T07:56:45.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f7385600640 1 -- 172.21.15.105:0/2629360028 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c044630 con 0x7f73880b1be0 2024-03-23T07:56:45.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.373+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7388142250 con 0x7f73880b1be0 2024-03-23T07:56:45.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.374+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73880cd500 con 0x7f73880b1be0 2024-03-23T07:56:45.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.375+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7388073390 con 0x7f73880b1be0 2024-03-23T07:56:45.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.375+0000 7f7385600640 1 -- 172.21.15.105:0/2629360028 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f737c02f050 con 0x7f73880b1be0 2024-03-23T07:56:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.376+0000 7f7385600640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f736806ac30 0x7f736806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.376+0000 7f7385600640 1 -- 172.21.15.105:0/2629360028 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f737c0ab2f0 con 0x7f73880b1be0 2024-03-23T07:56:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.377+0000 7f738d000640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f736806ac30 0x7f736806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.377+0000 7f738d000640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f736806ac30 0x7f736806d0f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f738806d6b0 tx=0x7f7374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:45.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.381+0000 7f7385600640 1 -- 172.21.15.105:0/2629360028 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f737c076270 con 0x7f73880b1be0 2024-03-23T07:56:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.615+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73880b2030 con 0x7f73880b1be0 2024-03-23T07:56:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.616+0000 7f7385600640 1 -- 172.21.15.105:0/2629360028 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7f73880b2030 con 0x7f73880b1be0 2024-03-23T07:56:45.617 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:45.617 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:56:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:56:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f736806ac30 msgr2=0x7f736806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f736806ac30 0x7f736806d0f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f738806d6b0 tx=0x7f7374008040 comp rx=0 tx=0).stop 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73880b1be0 msgr2=0x7f73880ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73880b1be0 0x7f73880ccfc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f737c009870 tx=0x7f737c0046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 shutdown_connections 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f736806ac30 0x7f736806d0f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7388160800 0x7f7388142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738813faa0 0x7f73881f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 --2- 172.21.15.105:0/2629360028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73880b1be0 0x7f73880ccfc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 >> 172.21.15.105:0/2629360028 conn(0x7f738806a3d0 msgr2=0x7f73880ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 shutdown_connections 2024-03-23T07:56:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.619+0000 7f7390400640 1 -- 172.21.15.105:0/2629360028 wait complete. 2024-03-23T07:56:45.636 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.843+0000 7f5181a00640 1 Processor -- start 2024-03-23T07:56:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.843+0000 7f5181a00640 1 -- start start 2024-03-23T07:56:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.844+0000 7f5181a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f517c0cde20 0x7f517c165360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.844+0000 7f5181a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c15b2b0 con 0x7f517c143930 2024-03-23T07:56:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.845+0000 7f5181a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c0cc6b0 con 0x7f517c143460 2024-03-23T07:56:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.845+0000 7f5181a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c0ce270 con 0x7f517c0cde20 2024-03-23T07:56:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.845+0000 7f5173400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f517c143930 0x7f517c0cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43028/0 (socket says 172.21.15.105:43028) 2024-03-23T07:56:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.845+0000 7f5173400640 1 -- 172.21.15.105:0/733081044 learned_addr learned my addr 172.21.15.105:0/733081044 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.845+0000 7f5181000640 1 --2- 172.21.15.105:0/733081044 >> v2:172.21.15.132:3300/0 conn(0x7f517c0cde20 0x7f517c165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.846+0000 7f517b400640 1 -- 172.21.15.105:0/733081044 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919537634 0 0) 0x7f517c15b2b0 con 0x7f517c143930 2024-03-23T07:56:45.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.846+0000 7f517b400640 1 -- 172.21.15.105:0/733081044 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5160003520 con 0x7f517c143930 2024-03-23T07:56:45.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.846+0000 7f5181000640 1 -- 172.21.15.105:0/733081044 >> v1:172.21.15.132:6789/0 conn(0x7f517c143460 legacy=0x7f517c15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.846+0000 7f5181000640 1 -- 172.21.15.105:0/733081044 >> v1:172.21.15.105:6789/0 conn(0x7f517c143930 legacy=0x7f517c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.846+0000 7f5181000640 1 -- 172.21.15.105:0/733081044 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c0cc950 con 0x7f517c0cde20 2024-03-23T07:56:45.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.847+0000 7f5181000640 1 --2- 172.21.15.105:0/733081044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c165360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f517402bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84745e2a55d5098c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:45.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f517b400640 1 -- 172.21.15.105:0/733081044 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517402cd80 con 0x7f517c0cde20 2024-03-23T07:56:45.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f517b400640 1 -- 172.21.15.105:0/733081044 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174034440 con 0x7f517c0cde20 2024-03-23T07:56:45.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f517b400640 1 -- 172.21.15.105:0/733081044 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740347d0 con 0x7f517c0cde20 2024-03-23T07:56:45.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f5181a00640 1 -- 172.21.15.105:0/733081044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 msgr2=0x7f517c165360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f5181a00640 1 --2- 172.21.15.105:0/733081044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c165360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f517402bfd0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f5181a00640 1 -- 172.21.15.105:0/733081044 shutdown_connections 2024-03-23T07:56:45.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f5181a00640 1 --2- 172.21.15.105:0/733081044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c165360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f5181a00640 1 -- 172.21.15.105:0/733081044 >> 172.21.15.105:0/733081044 conn(0x7f517c065df0 msgr2=0x7f517c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:45.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f5181a00640 1 -- 172.21.15.105:0/733081044 shutdown_connections 2024-03-23T07:56:45.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.848+0000 7f5181a00640 1 -- 172.21.15.105:0/733081044 wait complete. 2024-03-23T07:56:45.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.850+0000 7f5181a00640 1 Processor -- start 2024-03-23T07:56:45.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.850+0000 7f5181a00640 1 -- start start 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f517c143460 0x7f517c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f517c143930 0x7f517c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0ae6a0 con 0x7f517c143930 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0ae0e0 con 0x7f517c0cde20 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0ae3c0 con 0x7f517c143460 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f517c143930 0x7f517c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f517c143930 0x7f517c0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46550/0 (socket says 172.21.15.105:46550) 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.851+0000 7f5181000640 1 -- 172.21.15.105:0/1435946559 learned_addr learned my addr 172.21.15.105:0/1435946559 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5173400640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f517c143460 0x7f517c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f517be00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5181000640 1 -- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f517c143460 msgr2=0x7f517c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5181000640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f517c143460 0x7f517c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5181000640 1 -- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 msgr2=0x7f517c159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5181000640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5181000640 1 -- 172.21.15.105:0/1435946559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009a70 con 0x7f517c143930 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5173400640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f517c143460 0x7f517c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f517be00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5181000640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f517c143930 0x7f517c0b0530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f5174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5178c00640 1 -- 172.21.15.105:0/1435946559 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174040650 con 0x7f517c143930 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5178c00640 1 -- 172.21.15.105:0/1435946559 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51740407b0 con 0x7f517c143930 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c0aec20 con 0x7f517c143930 2024-03-23T07:56:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.852+0000 7f5178c00640 1 -- 172.21.15.105:0/1435946559 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174040b60 con 0x7f517c143930 2024-03-23T07:56:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.853+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c159400 con 0x7f517c143930 2024-03-23T07:56:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.854+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c159590 con 0x7f517c143930 2024-03-23T07:56:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.854+0000 7f5178c00640 1 -- 172.21.15.105:0/1435946559 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5174040cc0 con 0x7f517c143930 2024-03-23T07:56:45.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.856+0000 7f5178c00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f515806ac00 0x7f515806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:45.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.856+0000 7f5178c00640 1 -- 172.21.15.105:0/1435946559 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f51740ac420 con 0x7f517c143930 2024-03-23T07:56:45.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.856+0000 7f517be00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f515806ac00 0x7f515806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:45.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.857+0000 7f517be00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f515806ac00 0x7f515806d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b13e0 tx=0x7f516c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:45.860+0000 7f5178c00640 1 -- 172.21.15.105:0/1435946559 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f517c159590 con 0x7f517c143930 2024-03-23T07:56:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.065+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f517c159590 con 0x7f517c143930 2024-03-23T07:56:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.067+0000 7f5178c00640 1 -- 172.21.15.105:0/1435946559 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7f517c159590 con 0x7f517c143930 2024-03-23T07:56:46.068 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:46.068 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f515806ac00 msgr2=0x7f515806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f515806ac00 0x7f515806d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b13e0 tx=0x7f516c009040 comp rx=0 tx=0).stop 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f517c143930 msgr2=0x7f517c0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f517c143930 0x7f517c0b0530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f5174004a40 comp rx=0 tx=0).stop 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 shutdown_connections 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f517c143930 0x7f517c0b0530 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f515806ac00 0x7f515806d0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f517c143460 0x7f517c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 --2- 172.21.15.105:0/1435946559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f517c0cde20 0x7f517c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 >> 172.21.15.105:0/1435946559 conn(0x7f517c065df0 msgr2=0x7f517c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 shutdown_connections 2024-03-23T07:56:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.070+0000 7f5181a00640 1 -- 172.21.15.105:0/1435946559 wait complete. 2024-03-23T07:56:46.090 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T07:56:46.090 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:56:46.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.294+0000 7f7173000640 1 Processor -- start 2024-03-23T07:56:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.294+0000 7f7173000640 1 -- start start 2024-03-23T07:56:46.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7173000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c15b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7173000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f716c15d410 0x7f716c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7173000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c06c6f0 con 0x7f716c0b39b0 2024-03-23T07:56:46.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7173000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c0b4370 con 0x7f716c15d410 2024-03-23T07:56:46.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7173000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c06c9d0 con 0x7f716c15b560 2024-03-23T07:56:46.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7171000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f716c15d410 0x7f716c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f716b400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f716c0b39b0 0x7f716c15e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43034/0 (socket says 172.21.15.105:43034) 2024-03-23T07:56:46.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f716b400640 1 -- 172.21.15.105:0/500568743 learned_addr learned my addr 172.21.15.105:0/500568743 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:46.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7171000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f716c15d410 0x7f716c0b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46552/0 (socket says 172.21.15.105:46552) 2024-03-23T07:56:46.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7171000640 1 -- 172.21.15.105:0/500568743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 msgr2=0x7f716c15b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:46.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f716be00640 1 --2- 172.21.15.105:0/500568743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7171000640 1 --2- 172.21.15.105:0/500568743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c15b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f716aa00640 1 -- 172.21.15.105:0/500568743 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823793331 0 0) 0x7f716c06c6f0 con 0x7f716c0b39b0 2024-03-23T07:56:46.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7171000640 1 -- 172.21.15.105:0/500568743 >> v1:172.21.15.105:6789/0 conn(0x7f716c0b39b0 legacy=0x7f716c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.296+0000 7f7171000640 1 -- 172.21.15.105:0/500568743 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c06ccb0 con 0x7f716c15d410 2024-03-23T07:56:46.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.297+0000 7f716be00640 1 --2- 172.21.15.105:0/500568743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:46.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.298+0000 7f7171000640 1 --2- 172.21.15.105:0/500568743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c15d410 0x7f716c0b3e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7164009d90 tx=0x7f716402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecb6d501fbe3c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:46.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.298+0000 7f716aa00640 1 -- 172.21.15.105:0/500568743 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716402cc50 con 0x7f716c15d410 2024-03-23T07:56:46.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.298+0000 7f716aa00640 1 -- 172.21.15.105:0/500568743 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716402cdb0 con 0x7f716c15d410 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.298+0000 7f716aa00640 1 -- 172.21.15.105:0/500568743 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164034660 con 0x7f716c15d410 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.298+0000 7f7173000640 1 -- 172.21.15.105:0/500568743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c15d410 msgr2=0x7f716c0b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.298+0000 7f7173000640 1 --2- 172.21.15.105:0/500568743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c15d410 0x7f716c0b3e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7164009d90 tx=0x7f716402bea0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.299+0000 7f7173000640 1 -- 172.21.15.105:0/500568743 shutdown_connections 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.299+0000 7f7173000640 1 --2- 172.21.15.105:0/500568743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c15d410 0x7f716c0b3e30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.299+0000 7f7173000640 1 --2- 172.21.15.105:0/500568743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.299+0000 7f7173000640 1 -- 172.21.15.105:0/500568743 >> 172.21.15.105:0/500568743 conn(0x7f716c06a3d0 msgr2=0x7f716c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.299+0000 7f7173000640 1 -- 172.21.15.105:0/500568743 shutdown_connections 2024-03-23T07:56:46.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.299+0000 7f7173000640 1 -- 172.21.15.105:0/500568743 wait complete. 2024-03-23T07:56:46.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.300+0000 7f7173000640 1 Processor -- start 2024-03-23T07:56:46.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.300+0000 7f7173000640 1 -- start start 2024-03-23T07:56:46.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.301+0000 7f7173000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c0b39b0 0x7f716c13f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.301+0000 7f7173000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.301+0000 7f7173000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f716c15d410 0x7f716c159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.301+0000 7f7173000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c13f7d0 con 0x7f716c0b39b0 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.301+0000 7f7173000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b0900 con 0x7f716c15d410 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.301+0000 7f7173000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b1840 con 0x7f716c15b560 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c0b39b0 0x7f716c13f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c0b39b0 0x7f716c13f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46566/0 (socket says 172.21.15.105:46566) 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 -- 172.21.15.105:0/3869005017 learned_addr learned my addr 172.21.15.105:0/3869005017 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 -- 172.21.15.105:0/3869005017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 msgr2=0x7f716c0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 -- 172.21.15.105:0/3869005017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f716c15d410 msgr2=0x7f716c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f716c15d410 0x7f716c159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 -- 172.21.15.105:0/3869005017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164009a70 con 0x7f716c0b39b0 2024-03-23T07:56:46.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.302+0000 7f716be00640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c0b39b0 0x7f716c13f290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c7f0 tx=0x7f715c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:46.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.303+0000 7f7163e00640 1 -- 172.21.15.105:0/3869005017 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c007a30 con 0x7f716c0b39b0 2024-03-23T07:56:46.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.303+0000 7f7163e00640 1 -- 172.21.15.105:0/3869005017 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c007b90 con 0x7f716c0b39b0 2024-03-23T07:56:46.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.303+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c0b1b20 con 0x7f716c0b39b0 2024-03-23T07:56:46.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.303+0000 7f7163e00640 1 -- 172.21.15.105:0/3869005017 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00ce40 con 0x7f716c0b39b0 2024-03-23T07:56:46.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.303+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c0b20e0 con 0x7f716c0b39b0 2024-03-23T07:56:46.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.304+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c06ac20 con 0x7f716c0b39b0 2024-03-23T07:56:46.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.305+0000 7f7163e00640 1 -- 172.21.15.105:0/3869005017 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f715c0113f0 con 0x7f716c0b39b0 2024-03-23T07:56:46.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.306+0000 7f7163e00640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f714c06ac00 0x7f714c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.306+0000 7f7163e00640 1 -- 172.21.15.105:0/3869005017 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f715c089990 con 0x7f716c0b39b0 2024-03-23T07:56:46.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.306+0000 7f716b400640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f714c06ac00 0x7f714c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.307+0000 7f716b400640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f714c06ac00 0x7f714c06d0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7158007fd0 tx=0x7f7158007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:46.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.311+0000 7f7163e00640 1 -- 172.21.15.105:0/3869005017 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f715c054760 con 0x7f716c0b39b0 2024-03-23T07:56:46.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.500+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f716c06dd80 con 0x7f714c06ac00 2024-03-23T07:56:46.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.521+0000 7f7163e00640 1 -- 172.21.15.105:0/3869005017 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345105 (secure 0 0 0) 0x7f71580070d0 con 0x7f714c06ac00 2024-03-23T07:56:46.526 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:46.528 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:56:46.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f714c06ac00 msgr2=0x7f714c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f714c06ac00 0x7f714c06d0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7158007fd0 tx=0x7f7158007960 comp rx=0 tx=0).stop 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c0b39b0 msgr2=0x7f716c13f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c0b39b0 0x7f716c13f290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c7f0 tx=0x7f715c00cbf0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 shutdown_connections 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f716c15d410 0x7f716c159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f714c06ac00 0x7f714c06d0c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f716c15b560 0x7f716c0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 --2- 172.21.15.105:0/3869005017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f716c0b39b0 0x7f716c13f290 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.527+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 >> 172.21.15.105:0/3869005017 conn(0x7f716c06a3d0 msgr2=0x7f716c0cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.528+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 shutdown_connections 2024-03-23T07:56:46.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.528+0000 7f7173000640 1 -- 172.21.15.105:0/3869005017 wait complete. 2024-03-23T07:56:46.551 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-23T07:56:46.050598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2022002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004962"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995336+0000","last_change":"2024-03-23T07:56:17.227981+0000","last_active":"2024-03-23T07:56:21.995336+0000","last_peered":"2024-03-23T07:56:21.995336+0000","last_clean":"2024-03-23T07:56:21.995336+0000","last_became_active":"2024-03-23T07:56:17.227576+0000","last_became_peered":"2024-03-23T07:56:17.227576+0000","last_unstale":"2024-03-23T07:56:21.995336+0000","last_undegraded":"2024-03-23T07:56:21.995336+0000","last_fullsized":"2024-03-23T07:56:21.995336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001380+0000","last_change":"2024-03-23T07:56:17.730054+0000","last_active":"2024-03-23T07:56:22.001380+0000","last_peered":"2024-03-23T07:56:22.001380+0000","last_clean":"2024-03-23T07:56:22.001380+0000","last_became_active":"2024-03-23T07:56:17.729766+0000","last_became_peered":"2024-03-23T07:56:17.729766+0000","last_unstale":"2024-03-23T07:56:22.001380+0000","last_undegraded":"2024-03-23T07:56:22.001380+0000","last_fullsized":"2024-03-23T07:56:22.001380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:39.987411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.032954+0000","last_change":"2024-03-23T07:56:22.029375+0000","last_active":"2024-03-23T07:56:22.032954+0000","last_peered":"2024-03-23T07:56:22.032954+0000","last_clean":"2024-03-23T07:56:22.032954+0000","last_became_active":"2024-03-23T07:56:22.028383+0000","last_became_peered":"2024-03-23T07:56:22.028383+0000","last_unstale":"2024-03-23T07:56:22.032954+0000","last_undegraded":"2024-03-23T07:56:22.032954+0000","last_fullsized":"2024-03-23T07:56:22.032954+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:06.760193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001542+0000","last_change":"2024-03-23T07:56:17.217630+0000","last_active":"2024-03-23T07:56:22.001542+0000","last_peered":"2024-03-23T07:56:22.001542+0000","last_clean":"2024-03-23T07:56:22.001542+0000","last_became_active":"2024-03-23T07:56:17.217179+0000","last_became_peered":"2024-03-23T07:56:17.217179+0000","last_unstale":"2024-03-23T07:56:22.001542+0000","last_undegraded":"2024-03-23T07:56:22.001542+0000","last_fullsized":"2024-03-23T07:56:22.001542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:27:59.587867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004329+0000","last_change":"2024-03-23T07:56:17.734047+0000","last_active":"2024-03-23T07:56:22.004329+0000","last_peered":"2024-03-23T07:56:22.004329+0000","last_clean":"2024-03-23T07:56:22.004329+0000","last_became_active":"2024-03-23T07:56:17.733482+0000","last_became_peered":"2024-03-23T07:56:17.733482+0000","last_unstale":"2024-03-23T07:56:22.004329+0000","last_undegraded":"2024-03-23T07:56:22.004329+0000","last_fullsized":"2024-03-23T07:56:22.004329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:47:35.800634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093379+0000","last_change":"2024-03-23T07:56:22.012905+0000","last_active":"2024-03-23T07:56:22.093379+0000","last_peered":"2024-03-23T07:56:22.093379+0000","last_clean":"2024-03-23T07:56:22.093379+0000","last_became_active":"2024-03-23T07:56:22.012340+0000","last_became_peered":"2024-03-23T07:56:22.012340+0000","last_unstale":"2024-03-23T07:56:22.093379+0000","last_undegraded":"2024-03-23T07:56:22.093379+0000","last_fullsized":"2024-03-23T07:56:22.093379+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:43.862034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000002+0000","last_change":"2024-03-23T07:56:17.224614+0000","last_active":"2024-03-23T07:56:22.000002+0000","last_peered":"2024-03-23T07:56:22.000002+0000","last_clean":"2024-03-23T07:56:22.000002+0000","last_became_active":"2024-03-23T07:56:17.223463+0000","last_became_peered":"2024-03-23T07:56:17.223463+0000","last_unstale":"2024-03-23T07:56:22.000002+0000","last_undegraded":"2024-03-23T07:56:22.000002+0000","last_fullsized":"2024-03-23T07:56:22.000002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093607+0000","last_change":"2024-03-23T07:56:17.729551+0000","last_active":"2024-03-23T07:56:22.093607+0000","last_peered":"2024-03-23T07:56:22.093607+0000","last_clean":"2024-03-23T07:56:22.093607+0000","last_became_active":"2024-03-23T07:56:17.729295+0000","last_became_peered":"2024-03-23T07:56:17.729295+0000","last_unstale":"2024-03-23T07:56:22.093607+0000","last_undegraded":"2024-03-23T07:56:22.093607+0000","last_fullsized":"2024-03-23T07:56:22.093607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093533+0000","last_change":"2024-03-23T07:56:22.005574+0000","last_active":"2024-03-23T07:56:22.093533+0000","last_peered":"2024-03-23T07:56:22.093533+0000","last_clean":"2024-03-23T07:56:22.093533+0000","last_became_active":"2024-03-23T07:56:22.005169+0000","last_became_peered":"2024-03-23T07:56:22.005169+0000","last_unstale":"2024-03-23T07:56:22.093533+0000","last_undegraded":"2024-03-23T07:56:22.093533+0000","last_fullsized":"2024-03-23T07:56:22.093533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005504+0000","last_change":"2024-03-23T07:56:17.223235+0000","last_active":"2024-03-23T07:56:22.005504+0000","last_peered":"2024-03-23T07:56:22.005504+0000","last_clean":"2024-03-23T07:56:22.005504+0000","last_became_active":"2024-03-23T07:56:17.222925+0000","last_became_peered":"2024-03-23T07:56:17.222925+0000","last_unstale":"2024-03-23T07:56:22.005504+0000","last_undegraded":"2024-03-23T07:56:22.005504+0000","last_fullsized":"2024-03-23T07:56:22.005504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097254+0000","last_change":"2024-03-23T07:56:17.745979+0000","last_active":"2024-03-23T07:56:22.097254+0000","last_peered":"2024-03-23T07:56:22.097254+0000","last_clean":"2024-03-23T07:56:22.097254+0000","last_became_active":"2024-03-23T07:56:17.745702+0000","last_became_peered":"2024-03-23T07:56:17.745702+0000","last_unstale":"2024-03-23T07:56:22.097254+0000","last_undegraded":"2024-03-23T07:56:22.097254+0000","last_fullsized":"2024-03-23T07:56:22.097254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:07.760652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093472+0000","last_change":"2024-03-23T07:56:22.021092+0000","last_active":"2024-03-23T07:56:22.093472+0000","last_peered":"2024-03-23T07:56:22.093472+0000","last_clean":"2024-03-23T07:56:22.093472+0000","last_became_active":"2024-03-23T07:56:22.017901+0000","last_became_peered":"2024-03-23T07:56:22.017901+0000","last_unstale":"2024-03-23T07:56:22.093472+0000","last_undegraded":"2024-03-23T07:56:22.093472+0000","last_fullsized":"2024-03-23T07:56:22.093472+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097129+0000","last_change":"2024-03-23T07:56:17.733964+0000","last_active":"2024-03-23T07:56:22.097129+0000","last_peered":"2024-03-23T07:56:22.097129+0000","last_clean":"2024-03-23T07:56:22.097129+0000","last_became_active":"2024-03-23T07:56:17.733721+0000","last_became_peered":"2024-03-23T07:56:17.733721+0000","last_unstale":"2024-03-23T07:56:22.097129+0000","last_undegraded":"2024-03-23T07:56:22.097129+0000","last_fullsized":"2024-03-23T07:56:22.097129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004014+0000","last_change":"2024-03-23T07:56:17.227903+0000","last_active":"2024-03-23T07:56:22.004014+0000","last_peered":"2024-03-23T07:56:22.004014+0000","last_clean":"2024-03-23T07:56:22.004014+0000","last_became_active":"2024-03-23T07:56:17.227224+0000","last_became_peered":"2024-03-23T07:56:17.227224+0000","last_unstale":"2024-03-23T07:56:22.004014+0000","last_undegraded":"2024-03-23T07:56:22.004014+0000","last_fullsized":"2024-03-23T07:56:22.004014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097221+0000","last_change":"2024-03-23T07:56:22.023790+0000","last_active":"2024-03-23T07:56:22.097221+0000","last_peered":"2024-03-23T07:56:22.097221+0000","last_clean":"2024-03-23T07:56:22.097221+0000","last_became_active":"2024-03-23T07:56:22.022870+0000","last_became_peered":"2024-03-23T07:56:22.022870+0000","last_unstale":"2024-03-23T07:56:22.097221+0000","last_undegraded":"2024-03-23T07:56:22.097221+0000","last_fullsized":"2024-03-23T07:56:22.097221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:40.283526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014342+0000","last_change":"2024-03-23T07:56:17.734095+0000","last_active":"2024-03-23T07:56:22.014342+0000","last_peered":"2024-03-23T07:56:22.014342+0000","last_clean":"2024-03-23T07:56:22.014342+0000","last_became_active":"2024-03-23T07:56:17.733771+0000","last_became_peered":"2024-03-23T07:56:17.733771+0000","last_unstale":"2024-03-23T07:56:22.014342+0000","last_undegraded":"2024-03-23T07:56:22.014342+0000","last_fullsized":"2024-03-23T07:56:22.014342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002387+0000","last_change":"2024-03-23T07:56:17.221331+0000","last_active":"2024-03-23T07:56:22.002387+0000","last_peered":"2024-03-23T07:56:22.002387+0000","last_clean":"2024-03-23T07:56:22.002387+0000","last_became_active":"2024-03-23T07:56:17.221062+0000","last_became_peered":"2024-03-23T07:56:17.221062+0000","last_unstale":"2024-03-23T07:56:22.002387+0000","last_undegraded":"2024-03-23T07:56:22.002387+0000","last_fullsized":"2024-03-23T07:56:22.002387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096460+0000","last_change":"2024-03-23T07:56:22.028190+0000","last_active":"2024-03-23T07:56:22.096460+0000","last_peered":"2024-03-23T07:56:22.096460+0000","last_clean":"2024-03-23T07:56:22.096460+0000","last_became_active":"2024-03-23T07:56:22.027188+0000","last_became_peered":"2024-03-23T07:56:22.027188+0000","last_unstale":"2024-03-23T07:56:22.096460+0000","last_undegraded":"2024-03-23T07:56:22.096460+0000","last_fullsized":"2024-03-23T07:56:22.096460+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985338+0000","last_change":"2024-03-23T07:56:17.728823+0000","last_active":"2024-03-23T07:56:21.985338+0000","last_peered":"2024-03-23T07:56:21.985338+0000","last_clean":"2024-03-23T07:56:21.985338+0000","last_became_active":"2024-03-23T07:56:17.728504+0000","last_became_peered":"2024-03-23T07:56:17.728504+0000","last_unstale":"2024-03-23T07:56:21.985338+0000","last_undegraded":"2024-03-23T07:56:21.985338+0000","last_fullsized":"2024-03-23T07:56:21.985338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:28.882493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984446+0000","last_change":"2024-03-23T07:56:17.209430+0000","last_active":"2024-03-23T07:56:21.984446+0000","last_peered":"2024-03-23T07:56:21.984446+0000","last_clean":"2024-03-23T07:56:21.984446+0000","last_became_active":"2024-03-23T07:56:17.207873+0000","last_became_peered":"2024-03-23T07:56:17.207873+0000","last_unstale":"2024-03-23T07:56:21.984446+0000","last_undegraded":"2024-03-23T07:56:21.984446+0000","last_fullsized":"2024-03-23T07:56:21.984446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.033429+0000","last_change":"2024-03-23T07:56:22.031440+0000","last_active":"2024-03-23T07:56:22.033429+0000","last_peered":"2024-03-23T07:56:22.033429+0000","last_clean":"2024-03-23T07:56:22.033429+0000","last_became_active":"2024-03-23T07:56:22.030403+0000","last_became_peered":"2024-03-23T07:56:22.030403+0000","last_unstale":"2024-03-23T07:56:22.033429+0000","last_undegraded":"2024-03-23T07:56:22.033429+0000","last_fullsized":"2024-03-23T07:56:22.033429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:40.289743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097084+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097084+0000","last_peered":"2024-03-23T07:56:22.097084+0000","last_clean":"2024-03-23T07:56:22.097084+0000","last_became_active":"2024-03-23T07:56:17.728164+0000","last_became_peered":"2024-03-23T07:56:17.728164+0000","last_unstale":"2024-03-23T07:56:22.097084+0000","last_undegraded":"2024-03-23T07:56:22.097084+0000","last_fullsized":"2024-03-23T07:56:22.097084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:11:45.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998174+0000","last_change":"2024-03-23T07:56:17.211209+0000","last_active":"2024-03-23T07:56:21.998174+0000","last_peered":"2024-03-23T07:56:21.998174+0000","last_clean":"2024-03-23T07:56:21.998174+0000","last_became_active":"2024-03-23T07:56:17.210792+0000","last_became_peered":"2024-03-23T07:56:17.210792+0000","last_unstale":"2024-03-23T07:56:21.998174+0000","last_undegraded":"2024-03-23T07:56:21.998174+0000","last_fullsized":"2024-03-23T07:56:21.998174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093682+0000","last_change":"2024-03-23T07:56:22.016254+0000","last_active":"2024-03-23T07:56:22.093682+0000","last_peered":"2024-03-23T07:56:22.093682+0000","last_clean":"2024-03-23T07:56:22.093682+0000","last_became_active":"2024-03-23T07:56:22.015890+0000","last_became_peered":"2024-03-23T07:56:22.015890+0000","last_unstale":"2024-03-23T07:56:22.093682+0000","last_undegraded":"2024-03-23T07:56:22.093682+0000","last_fullsized":"2024-03-23T07:56:22.093682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:15.477490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992721+0000","last_change":"2024-03-23T07:56:17.726007+0000","last_active":"2024-03-23T07:56:21.992721+0000","last_peered":"2024-03-23T07:56:21.992721+0000","last_clean":"2024-03-23T07:56:21.992721+0000","last_became_active":"2024-03-23T07:56:17.723965+0000","last_became_peered":"2024-03-23T07:56:17.723965+0000","last_unstale":"2024-03-23T07:56:21.992721+0000","last_undegraded":"2024-03-23T07:56:21.992721+0000","last_fullsized":"2024-03-23T07:56:21.992721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004697+0000","last_change":"2024-03-23T07:56:17.225692+0000","last_active":"2024-03-23T07:56:22.004697+0000","last_peered":"2024-03-23T07:56:22.004697+0000","last_clean":"2024-03-23T07:56:22.004697+0000","last_became_active":"2024-03-23T07:56:17.225384+0000","last_became_peered":"2024-03-23T07:56:17.225384+0000","last_unstale":"2024-03-23T07:56:22.004697+0000","last_undegraded":"2024-03-23T07:56:22.004697+0000","last_fullsized":"2024-03-23T07:56:22.004697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:21.655121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093432+0000","last_change":"2024-03-23T07:56:22.007586+0000","last_active":"2024-03-23T07:56:22.093432+0000","last_peered":"2024-03-23T07:56:22.093432+0000","last_clean":"2024-03-23T07:56:22.093432+0000","last_became_active":"2024-03-23T07:56:22.007159+0000","last_became_peered":"2024-03-23T07:56:22.007159+0000","last_unstale":"2024-03-23T07:56:22.093432+0000","last_undegraded":"2024-03-23T07:56:22.093432+0000","last_fullsized":"2024-03-23T07:56:22.093432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995105+0000","last_change":"2024-03-23T07:56:17.722742+0000","last_active":"2024-03-23T07:56:21.995105+0000","last_peered":"2024-03-23T07:56:21.995105+0000","last_clean":"2024-03-23T07:56:21.995105+0000","last_became_active":"2024-03-23T07:56:17.722246+0000","last_became_peered":"2024-03-23T07:56:17.722246+0000","last_unstale":"2024-03-23T07:56:21.995105+0000","last_undegraded":"2024-03-23T07:56:21.995105+0000","last_fullsized":"2024-03-23T07:56:21.995105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994508+0000","last_change":"2024-03-23T07:56:17.212269+0000","last_active":"2024-03-23T07:56:21.994508+0000","last_peered":"2024-03-23T07:56:21.994508+0000","last_clean":"2024-03-23T07:56:21.994508+0000","last_became_active":"2024-03-23T07:56:17.211965+0000","last_became_peered":"2024-03-23T07:56:17.211965+0000","last_unstale":"2024-03-23T07:56:21.994508+0000","last_undegraded":"2024-03-23T07:56:21.994508+0000","last_fullsized":"2024-03-23T07:56:21.994508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015727+0000","last_change":"2024-03-23T07:56:22.009793+0000","last_active":"2024-03-23T07:56:22.015727+0000","last_peered":"2024-03-23T07:56:22.015727+0000","last_clean":"2024-03-23T07:56:22.015727+0000","last_became_active":"2024-03-23T07:56:22.009417+0000","last_became_peered":"2024-03-23T07:56:22.009417+0000","last_unstale":"2024-03-23T07:56:22.015727+0000","last_undegraded":"2024-03-23T07:56:22.015727+0000","last_fullsized":"2024-03-23T07:56:22.015727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003091+0000","last_change":"2024-03-23T07:56:17.726582+0000","last_active":"2024-03-23T07:56:22.003091+0000","last_peered":"2024-03-23T07:56:22.003091+0000","last_clean":"2024-03-23T07:56:22.003091+0000","last_became_active":"2024-03-23T07:56:17.726338+0000","last_became_peered":"2024-03-23T07:56:17.726338+0000","last_unstale":"2024-03-23T07:56:22.003091+0000","last_undegraded":"2024-03-23T07:56:22.003091+0000","last_fullsized":"2024-03-23T07:56:22.003091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004594+0000","last_change":"2024-03-23T07:56:17.215985+0000","last_active":"2024-03-23T07:56:22.004594+0000","last_peered":"2024-03-23T07:56:22.004594+0000","last_clean":"2024-03-23T07:56:22.004594+0000","last_became_active":"2024-03-23T07:56:17.215709+0000","last_became_peered":"2024-03-23T07:56:17.215709+0000","last_unstale":"2024-03-23T07:56:22.004594+0000","last_undegraded":"2024-03-23T07:56:22.004594+0000","last_fullsized":"2024-03-23T07:56:22.004594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028361+0000","last_change":"2024-03-23T07:56:22.026672+0000","last_active":"2024-03-23T07:56:22.028361+0000","last_peered":"2024-03-23T07:56:22.028361+0000","last_clean":"2024-03-23T07:56:22.028361+0000","last_became_active":"2024-03-23T07:56:22.025340+0000","last_became_peered":"2024-03-23T07:56:22.025340+0000","last_unstale":"2024-03-23T07:56:22.028361+0000","last_undegraded":"2024-03-23T07:56:22.028361+0000","last_fullsized":"2024-03-23T07:56:22.028361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:59.226182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093663+0000","last_change":"2024-03-23T07:56:17.723057+0000","last_active":"2024-03-23T07:56:22.093663+0000","last_peered":"2024-03-23T07:56:22.093663+0000","last_clean":"2024-03-23T07:56:22.093663+0000","last_became_active":"2024-03-23T07:56:17.722787+0000","last_became_peered":"2024-03-23T07:56:17.722787+0000","last_unstale":"2024-03-23T07:56:22.093663+0000","last_undegraded":"2024-03-23T07:56:22.093663+0000","last_fullsized":"2024-03-23T07:56:22.093663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992051+0000","last_change":"2024-03-23T07:56:17.217582+0000","last_active":"2024-03-23T07:56:21.992051+0000","last_peered":"2024-03-23T07:56:21.992051+0000","last_clean":"2024-03-23T07:56:21.992051+0000","last_became_active":"2024-03-23T07:56:17.217205+0000","last_became_peered":"2024-03-23T07:56:17.217205+0000","last_unstale":"2024-03-23T07:56:21.992051+0000","last_undegraded":"2024-03-23T07:56:21.992051+0000","last_fullsized":"2024-03-23T07:56:21.992051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:23:32.182551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.027442+0000","last_change":"2024-03-23T07:56:22.023281+0000","last_active":"2024-03-23T07:56:22.027442+0000","last_peered":"2024-03-23T07:56:22.027442+0000","last_clean":"2024-03-23T07:56:22.027442+0000","last_became_active":"2024-03-23T07:56:22.022756+0000","last_became_peered":"2024-03-23T07:56:22.022756+0000","last_unstale":"2024-03-23T07:56:22.027442+0000","last_undegraded":"2024-03-23T07:56:22.027442+0000","last_fullsized":"2024-03-23T07:56:22.027442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096425+0000","last_change":"2024-03-23T07:56:17.740008+0000","last_active":"2024-03-23T07:56:22.096425+0000","last_peered":"2024-03-23T07:56:22.096425+0000","last_clean":"2024-03-23T07:56:22.096425+0000","last_became_active":"2024-03-23T07:56:17.739785+0000","last_became_peered":"2024-03-23T07:56:17.739785+0000","last_unstale":"2024-03-23T07:56:22.096425+0000","last_undegraded":"2024-03-23T07:56:22.096425+0000","last_fullsized":"2024-03-23T07:56:22.096425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003892+0000","last_change":"2024-03-23T07:56:17.221620+0000","last_active":"2024-03-23T07:56:22.003892+0000","last_peered":"2024-03-23T07:56:22.003892+0000","last_clean":"2024-03-23T07:56:22.003892+0000","last_became_active":"2024-03-23T07:56:17.221312+0000","last_became_peered":"2024-03-23T07:56:17.221312+0000","last_unstale":"2024-03-23T07:56:22.003892+0000","last_undegraded":"2024-03-23T07:56:22.003892+0000","last_fullsized":"2024-03-23T07:56:22.003892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:47.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.024787+0000","last_change":"2024-03-23T07:56:22.018139+0000","last_active":"2024-03-23T07:56:22.024787+0000","last_peered":"2024-03-23T07:56:22.024787+0000","last_clean":"2024-03-23T07:56:22.024787+0000","last_became_active":"2024-03-23T07:56:22.017722+0000","last_became_peered":"2024-03-23T07:56:22.017722+0000","last_unstale":"2024-03-23T07:56:22.024787+0000","last_undegraded":"2024-03-23T07:56:22.024787+0000","last_fullsized":"2024-03-23T07:56:22.024787+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983276+0000","last_change":"2024-03-23T07:56:17.729485+0000","last_active":"2024-03-23T07:56:21.983276+0000","last_peered":"2024-03-23T07:56:21.983276+0000","last_clean":"2024-03-23T07:56:21.983276+0000","last_became_active":"2024-03-23T07:56:17.729199+0000","last_became_peered":"2024-03-23T07:56:17.729199+0000","last_unstale":"2024-03-23T07:56:21.983276+0000","last_undegraded":"2024-03-23T07:56:21.983276+0000","last_fullsized":"2024-03-23T07:56:21.983276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:30:59.262864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014955+0000","last_change":"2024-03-23T07:56:17.222373+0000","last_active":"2024-03-23T07:56:22.014955+0000","last_peered":"2024-03-23T07:56:22.014955+0000","last_clean":"2024-03-23T07:56:22.014955+0000","last_became_active":"2024-03-23T07:56:17.222028+0000","last_became_peered":"2024-03-23T07:56:17.222028+0000","last_unstale":"2024-03-23T07:56:22.014955+0000","last_undegraded":"2024-03-23T07:56:22.014955+0000","last_fullsized":"2024-03-23T07:56:22.014955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.023027+0000","last_change":"2024-03-23T07:56:22.018041+0000","last_active":"2024-03-23T07:56:22.023027+0000","last_peered":"2024-03-23T07:56:22.023027+0000","last_clean":"2024-03-23T07:56:22.023027+0000","last_became_active":"2024-03-23T07:56:22.017588+0000","last_became_peered":"2024-03-23T07:56:22.017588+0000","last_unstale":"2024-03-23T07:56:22.023027+0000","last_undegraded":"2024-03-23T07:56:22.023027+0000","last_fullsized":"2024-03-23T07:56:22.023027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001772+0000","last_change":"2024-03-23T07:56:17.724698+0000","last_active":"2024-03-23T07:56:22.001772+0000","last_peered":"2024-03-23T07:56:22.001772+0000","last_clean":"2024-03-23T07:56:22.001772+0000","last_became_active":"2024-03-23T07:56:17.724382+0000","last_became_peered":"2024-03-23T07:56:17.724382+0000","last_unstale":"2024-03-23T07:56:22.001772+0000","last_undegraded":"2024-03-23T07:56:22.001772+0000","last_fullsized":"2024-03-23T07:56:22.001772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002505+0000","last_change":"2024-03-23T07:56:17.223691+0000","last_active":"2024-03-23T07:56:22.002505+0000","last_peered":"2024-03-23T07:56:22.002505+0000","last_clean":"2024-03-23T07:56:22.002505+0000","last_became_active":"2024-03-23T07:56:17.222061+0000","last_became_peered":"2024-03-23T07:56:17.222061+0000","last_unstale":"2024-03-23T07:56:22.002505+0000","last_undegraded":"2024-03-23T07:56:22.002505+0000","last_fullsized":"2024-03-23T07:56:22.002505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:00.798128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471272+0000","last_change":"2024-03-23T07:56:22.011743+0000","last_active":"2024-03-23T07:56:22.471272+0000","last_peered":"2024-03-23T07:56:22.471272+0000","last_clean":"2024-03-23T07:56:22.471272+0000","last_became_active":"2024-03-23T07:56:22.011280+0000","last_became_peered":"2024-03-23T07:56:22.011280+0000","last_unstale":"2024-03-23T07:56:22.471272+0000","last_undegraded":"2024-03-23T07:56:22.471272+0000","last_fullsized":"2024-03-23T07:56:22.471272+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:05.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988647+0000","last_change":"2024-03-23T07:56:17.722136+0000","last_active":"2024-03-23T07:56:21.988647+0000","last_peered":"2024-03-23T07:56:21.988647+0000","last_clean":"2024-03-23T07:56:21.988647+0000","last_became_active":"2024-03-23T07:56:17.720479+0000","last_became_peered":"2024-03-23T07:56:17.720479+0000","last_unstale":"2024-03-23T07:56:21.988647+0000","last_undegraded":"2024-03-23T07:56:21.988647+0000","last_fullsized":"2024-03-23T07:56:21.988647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003825+0000","last_change":"2024-03-23T07:56:17.227266+0000","last_active":"2024-03-23T07:56:22.003825+0000","last_peered":"2024-03-23T07:56:22.003825+0000","last_clean":"2024-03-23T07:56:22.003825+0000","last_became_active":"2024-03-23T07:56:17.226775+0000","last_became_peered":"2024-03-23T07:56:17.226775+0000","last_unstale":"2024-03-23T07:56:22.003825+0000","last_undegraded":"2024-03-23T07:56:22.003825+0000","last_fullsized":"2024-03-23T07:56:22.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005759+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:56:22.005759+0000","last_peered":"2024-03-23T07:56:22.005759+0000","last_clean":"2024-03-23T07:56:22.005759+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:56:22.005759+0000","last_undegraded":"2024-03-23T07:56:22.005759+0000","last_fullsized":"2024-03-23T07:56:22.005759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097280+0000","last_change":"2024-03-23T07:56:22.024202+0000","last_active":"2024-03-23T07:56:22.097280+0000","last_peered":"2024-03-23T07:56:22.097280+0000","last_clean":"2024-03-23T07:56:22.097280+0000","last_became_active":"2024-03-23T07:56:22.018941+0000","last_became_peered":"2024-03-23T07:56:22.018941+0000","last_unstale":"2024-03-23T07:56:22.097280+0000","last_undegraded":"2024-03-23T07:56:22.097280+0000","last_fullsized":"2024-03-23T07:56:22.097280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:25.917706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097267+0000","last_change":"2024-03-23T07:56:17.734848+0000","last_active":"2024-03-23T07:56:22.097267+0000","last_peered":"2024-03-23T07:56:22.097267+0000","last_clean":"2024-03-23T07:56:22.097267+0000","last_became_active":"2024-03-23T07:56:17.733233+0000","last_became_peered":"2024-03-23T07:56:17.733233+0000","last_unstale":"2024-03-23T07:56:22.097267+0000","last_undegraded":"2024-03-23T07:56:22.097267+0000","last_fullsized":"2024-03-23T07:56:22.097267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003338+0000","last_change":"2024-03-23T07:56:17.215068+0000","last_active":"2024-03-23T07:56:22.003338+0000","last_peered":"2024-03-23T07:56:22.003338+0000","last_clean":"2024-03-23T07:56:22.003338+0000","last_became_active":"2024-03-23T07:56:17.214726+0000","last_became_peered":"2024-03-23T07:56:17.214726+0000","last_unstale":"2024-03-23T07:56:22.003338+0000","last_undegraded":"2024-03-23T07:56:22.003338+0000","last_fullsized":"2024-03-23T07:56:22.003338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093494+0000","last_change":"2024-03-23T07:56:22.023739+0000","last_active":"2024-03-23T07:56:22.093494+0000","last_peered":"2024-03-23T07:56:22.093494+0000","last_clean":"2024-03-23T07:56:22.093494+0000","last_became_active":"2024-03-23T07:56:22.023366+0000","last_became_peered":"2024-03-23T07:56:22.023366+0000","last_unstale":"2024-03-23T07:56:22.093494+0000","last_undegraded":"2024-03-23T07:56:22.093494+0000","last_fullsized":"2024-03-23T07:56:22.093494+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:13.143472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096511+0000","last_change":"2024-03-23T07:56:17.737947+0000","last_active":"2024-03-23T07:56:22.096511+0000","last_peered":"2024-03-23T07:56:22.096511+0000","last_clean":"2024-03-23T07:56:22.096511+0000","last_became_active":"2024-03-23T07:56:17.733346+0000","last_became_peered":"2024-03-23T07:56:17.733346+0000","last_unstale":"2024-03-23T07:56:22.096511+0000","last_undegraded":"2024-03-23T07:56:22.096511+0000","last_fullsized":"2024-03-23T07:56:22.096511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998748+0000","last_change":"2024-03-23T07:56:17.221283+0000","last_active":"2024-03-23T07:56:21.998748+0000","last_peered":"2024-03-23T07:56:21.998748+0000","last_clean":"2024-03-23T07:56:21.998748+0000","last_became_active":"2024-03-23T07:56:17.219961+0000","last_became_peered":"2024-03-23T07:56:17.219961+0000","last_unstale":"2024-03-23T07:56:21.998748+0000","last_undegraded":"2024-03-23T07:56:21.998748+0000","last_fullsized":"2024-03-23T07:56:21.998748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:04:11.860279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.470374+0000","last_change":"2024-03-23T07:56:22.016551+0000","last_active":"2024-03-23T07:56:22.470374+0000","last_peered":"2024-03-23T07:56:22.470374+0000","last_clean":"2024-03-23T07:56:22.470374+0000","last_became_active":"2024-03-23T07:56:22.016136+0000","last_became_peered":"2024-03-23T07:56:22.016136+0000","last_unstale":"2024-03-23T07:56:22.470374+0000","last_undegraded":"2024-03-23T07:56:22.470374+0000","last_fullsized":"2024-03-23T07:56:22.470374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:36.873288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096655+0000","last_change":"2024-03-23T07:56:17.743611+0000","last_active":"2024-03-23T07:56:22.096655+0000","last_peered":"2024-03-23T07:56:22.096655+0000","last_clean":"2024-03-23T07:56:22.096655+0000","last_became_active":"2024-03-23T07:56:17.742926+0000","last_became_peered":"2024-03-23T07:56:17.742926+0000","last_unstale":"2024-03-23T07:56:22.096655+0000","last_undegraded":"2024-03-23T07:56:22.096655+0000","last_fullsized":"2024-03-23T07:56:22.096655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994328+0000","last_change":"2024-03-23T07:56:17.219709+0000","last_active":"2024-03-23T07:56:21.994328+0000","last_peered":"2024-03-23T07:56:21.994328+0000","last_clean":"2024-03-23T07:56:21.994328+0000","last_became_active":"2024-03-23T07:56:17.218465+0000","last_became_peered":"2024-03-23T07:56:17.218465+0000","last_unstale":"2024-03-23T07:56:21.994328+0000","last_undegraded":"2024-03-23T07:56:21.994328+0000","last_fullsized":"2024-03-23T07:56:21.994328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096964+0000","last_change":"2024-03-23T07:56:22.017346+0000","last_active":"2024-03-23T07:56:22.096964+0000","last_peered":"2024-03-23T07:56:22.096964+0000","last_clean":"2024-03-23T07:56:22.096964+0000","last_became_active":"2024-03-23T07:56:22.016949+0000","last_became_peered":"2024-03-23T07:56:22.016949+0000","last_unstale":"2024-03-23T07:56:22.096964+0000","last_undegraded":"2024-03-23T07:56:22.096964+0000","last_fullsized":"2024-03-23T07:56:22.096964+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:45.299178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093733+0000","last_change":"2024-03-23T07:56:17.742996+0000","last_active":"2024-03-23T07:56:22.093733+0000","last_peered":"2024-03-23T07:56:22.093733+0000","last_clean":"2024-03-23T07:56:22.093733+0000","last_became_active":"2024-03-23T07:56:17.742660+0000","last_became_peered":"2024-03-23T07:56:17.742660+0000","last_unstale":"2024-03-23T07:56:22.093733+0000","last_undegraded":"2024-03-23T07:56:22.093733+0000","last_fullsized":"2024-03-23T07:56:22.093733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004353+0000","last_change":"2024-03-23T07:56:17.215901+0000","last_active":"2024-03-23T07:56:22.004353+0000","last_peered":"2024-03-23T07:56:22.004353+0000","last_clean":"2024-03-23T07:56:22.004353+0000","last_became_active":"2024-03-23T07:56:17.215638+0000","last_became_peered":"2024-03-23T07:56:17.215638+0000","last_unstale":"2024-03-23T07:56:22.004353+0000","last_undegraded":"2024-03-23T07:56:22.004353+0000","last_fullsized":"2024-03-23T07:56:22.004353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093529+0000","last_change":"2024-03-23T07:56:22.010512+0000","last_active":"2024-03-23T07:56:22.093529+0000","last_peered":"2024-03-23T07:56:22.093529+0000","last_clean":"2024-03-23T07:56:22.093529+0000","last_became_active":"2024-03-23T07:56:22.009577+0000","last_became_peered":"2024-03-23T07:56:22.009577+0000","last_unstale":"2024-03-23T07:56:22.093529+0000","last_undegraded":"2024-03-23T07:56:22.093529+0000","last_fullsized":"2024-03-23T07:56:22.093529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:24.844391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097072+0000","last_change":"2024-03-23T07:56:17.732277+0000","last_active":"2024-03-23T07:56:22.097072+0000","last_peered":"2024-03-23T07:56:22.097072+0000","last_clean":"2024-03-23T07:56:22.097072+0000","last_became_active":"2024-03-23T07:56:17.731487+0000","last_became_peered":"2024-03-23T07:56:17.731487+0000","last_unstale":"2024-03-23T07:56:22.097072+0000","last_undegraded":"2024-03-23T07:56:22.097072+0000","last_fullsized":"2024-03-23T07:56:22.097072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:39.462067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985538+0000","last_change":"2024-03-23T07:56:17.211010+0000","last_active":"2024-03-23T07:56:21.985538+0000","last_peered":"2024-03-23T07:56:21.985538+0000","last_clean":"2024-03-23T07:56:21.985538+0000","last_became_active":"2024-03-23T07:56:17.210710+0000","last_became_peered":"2024-03-23T07:56:17.210710+0000","last_unstale":"2024-03-23T07:56:21.985538+0000","last_undegraded":"2024-03-23T07:56:21.985538+0000","last_fullsized":"2024-03-23T07:56:21.985538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471596+0000","last_change":"2024-03-23T07:56:22.019325+0000","last_active":"2024-03-23T07:56:22.471596+0000","last_peered":"2024-03-23T07:56:22.471596+0000","last_clean":"2024-03-23T07:56:22.471596+0000","last_became_active":"2024-03-23T07:56:22.016793+0000","last_became_peered":"2024-03-23T07:56:22.016793+0000","last_unstale":"2024-03-23T07:56:22.471596+0000","last_undegraded":"2024-03-23T07:56:22.471596+0000","last_fullsized":"2024-03-23T07:56:22.471596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:08:59.171123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093481+0000","last_change":"2024-03-23T07:56:17.716144+0000","last_active":"2024-03-23T07:56:22.093481+0000","last_peered":"2024-03-23T07:56:22.093481+0000","last_clean":"2024-03-23T07:56:22.093481+0000","last_became_active":"2024-03-23T07:56:17.715899+0000","last_became_peered":"2024-03-23T07:56:17.715899+0000","last_unstale":"2024-03-23T07:56:22.093481+0000","last_undegraded":"2024-03-23T07:56:22.093481+0000","last_fullsized":"2024-03-23T07:56:22.093481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005380+0000","last_change":"2024-03-23T07:56:17.226828+0000","last_active":"2024-03-23T07:56:22.005380+0000","last_peered":"2024-03-23T07:56:22.005380+0000","last_clean":"2024-03-23T07:56:22.005380+0000","last_became_active":"2024-03-23T07:56:17.226551+0000","last_became_peered":"2024-03-23T07:56:17.226551+0000","last_unstale":"2024-03-23T07:56:22.005380+0000","last_undegraded":"2024-03-23T07:56:22.005380+0000","last_fullsized":"2024-03-23T07:56:22.005380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:03.843216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471755+0000","last_change":"2024-03-23T07:56:22.019606+0000","last_active":"2024-03-23T07:56:22.471755+0000","last_peered":"2024-03-23T07:56:22.471755+0000","last_clean":"2024-03-23T07:56:22.471755+0000","last_became_active":"2024-03-23T07:56:22.019280+0000","last_became_peered":"2024-03-23T07:56:22.019280+0000","last_unstale":"2024-03-23T07:56:22.471755+0000","last_undegraded":"2024-03-23T07:56:22.471755+0000","last_fullsized":"2024-03-23T07:56:22.471755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:28.706338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097162+0000","last_change":"2024-03-23T07:56:17.746537+0000","last_active":"2024-03-23T07:56:22.097162+0000","last_peered":"2024-03-23T07:56:22.097162+0000","last_clean":"2024-03-23T07:56:22.097162+0000","last_became_active":"2024-03-23T07:56:17.746054+0000","last_became_peered":"2024-03-23T07:56:17.746054+0000","last_unstale":"2024-03-23T07:56:22.097162+0000","last_undegraded":"2024-03-23T07:56:22.097162+0000","last_fullsized":"2024-03-23T07:56:22.097162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:09.358882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003661+0000","last_change":"2024-03-23T07:56:17.225527+0000","last_active":"2024-03-23T07:56:22.003661+0000","last_peered":"2024-03-23T07:56:22.003661+0000","last_clean":"2024-03-23T07:56:22.003661+0000","last_became_active":"2024-03-23T07:56:17.224860+0000","last_became_peered":"2024-03-23T07:56:17.224860+0000","last_unstale":"2024-03-23T07:56:22.003661+0000","last_undegraded":"2024-03-23T07:56:22.003661+0000","last_fullsized":"2024-03-23T07:56:22.003661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:26.692314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028094+0000","last_change":"2024-03-23T07:56:22.023659+0000","last_active":"2024-03-23T07:56:22.028094+0000","last_peered":"2024-03-23T07:56:22.028094+0000","last_clean":"2024-03-23T07:56:22.028094+0000","last_became_active":"2024-03-23T07:56:22.023230+0000","last_became_peered":"2024-03-23T07:56:22.023230+0000","last_unstale":"2024-03-23T07:56:22.028094+0000","last_undegraded":"2024-03-23T07:56:22.028094+0000","last_fullsized":"2024-03-23T07:56:22.028094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002303+0000","last_change":"2024-03-23T07:56:17.729587+0000","last_active":"2024-03-23T07:56:22.002303+0000","last_peered":"2024-03-23T07:56:22.002303+0000","last_clean":"2024-03-23T07:56:22.002303+0000","last_became_active":"2024-03-23T07:56:17.727873+0000","last_became_peered":"2024-03-23T07:56:17.727873+0000","last_unstale":"2024-03-23T07:56:22.002303+0000","last_undegraded":"2024-03-23T07:56:22.002303+0000","last_fullsized":"2024-03-23T07:56:22.002303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017227+0000","last_change":"2024-03-23T07:56:17.215408+0000","last_active":"2024-03-23T07:56:22.017227+0000","last_peered":"2024-03-23T07:56:22.017227+0000","last_clean":"2024-03-23T07:56:22.017227+0000","last_became_active":"2024-03-23T07:56:17.215155+0000","last_became_peered":"2024-03-23T07:56:17.215155+0000","last_unstale":"2024-03-23T07:56:22.017227+0000","last_undegraded":"2024-03-23T07:56:22.017227+0000","last_fullsized":"2024-03-23T07:56:22.017227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097238+0000","last_change":"2024-03-23T07:56:22.010493+0000","last_active":"2024-03-23T07:56:22.097238+0000","last_peered":"2024-03-23T07:56:22.097238+0000","last_clean":"2024-03-23T07:56:22.097238+0000","last_became_active":"2024-03-23T07:56:22.010048+0000","last_became_peered":"2024-03-23T07:56:22.010048+0000","last_unstale":"2024-03-23T07:56:22.097238+0000","last_undegraded":"2024-03-23T07:56:22.097238+0000","last_fullsized":"2024-03-23T07:56:22.097238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:17.490111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096554+0000","last_change":"2024-03-23T07:56:17.735587+0000","last_active":"2024-03-23T07:56:22.096554+0000","last_peered":"2024-03-23T07:56:22.096554+0000","last_clean":"2024-03-23T07:56:22.096554+0000","last_became_active":"2024-03-23T07:56:17.735319+0000","last_became_peered":"2024-03-23T07:56:17.735319+0000","last_unstale":"2024-03-23T07:56:22.096554+0000","last_undegraded":"2024-03-23T07:56:22.096554+0000","last_fullsized":"2024-03-23T07:56:22.096554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:36.310952+0000","last_change":"2024-03-23T07:56:17.217671+0000","last_active":"2024-03-23T07:56:36.310952+0000","last_peered":"2024-03-23T07:56:36.310952+0000","last_clean":"2024-03-23T07:56:36.310952+0000","last_became_active":"2024-03-23T07:56:17.217332+0000","last_became_peered":"2024-03-23T07:56:17.217332+0000","last_unstale":"2024-03-23T07:56:36.310952+0000","last_undegraded":"2024-03-23T07:56:36.310952+0000","last_fullsized":"2024-03-23T07:56:36.310952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:40:30.911275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018197+0000","last_change":"2024-03-23T07:56:22.013208+0000","last_active":"2024-03-23T07:56:22.018197+0000","last_peered":"2024-03-23T07:56:22.018197+0000","last_clean":"2024-03-23T07:56:22.018197+0000","last_became_active":"2024-03-23T07:56:22.012577+0000","last_became_peered":"2024-03-23T07:56:22.012577+0000","last_unstale":"2024-03-23T07:56:22.018197+0000","last_undegraded":"2024-03-23T07:56:22.018197+0000","last_fullsized":"2024-03-23T07:56:22.018197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097206+0000","last_change":"2024-03-23T07:56:17.735251+0000","last_active":"2024-03-23T07:56:22.097206+0000","last_peered":"2024-03-23T07:56:22.097206+0000","last_clean":"2024-03-23T07:56:22.097206+0000","last_became_active":"2024-03-23T07:56:17.731173+0000","last_became_peered":"2024-03-23T07:56:17.731173+0000","last_unstale":"2024-03-23T07:56:22.097206+0000","last_undegraded":"2024-03-23T07:56:22.097206+0000","last_fullsized":"2024-03-23T07:56:22.097206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:54.379935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018228+0000","last_change":"2024-03-23T07:56:17.216512+0000","last_active":"2024-03-23T07:56:22.018228+0000","last_peered":"2024-03-23T07:56:22.018228+0000","last_clean":"2024-03-23T07:56:22.018228+0000","last_became_active":"2024-03-23T07:56:17.215743+0000","last_became_peered":"2024-03-23T07:56:17.215743+0000","last_unstale":"2024-03-23T07:56:22.018228+0000","last_undegraded":"2024-03-23T07:56:22.018228+0000","last_fullsized":"2024-03-23T07:56:22.018228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093611+0000","last_change":"2024-03-23T07:56:22.014702+0000","last_active":"2024-03-23T07:56:22.093611+0000","last_peered":"2024-03-23T07:56:22.093611+0000","last_clean":"2024-03-23T07:56:22.093611+0000","last_became_active":"2024-03-23T07:56:22.014290+0000","last_became_peered":"2024-03-23T07:56:22.014290+0000","last_unstale":"2024-03-23T07:56:22.093611+0000","last_undegraded":"2024-03-23T07:56:22.093611+0000","last_fullsized":"2024-03-23T07:56:22.093611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005107+0000","last_change":"2024-03-23T07:56:17.725552+0000","last_active":"2024-03-23T07:56:22.005107+0000","last_peered":"2024-03-23T07:56:22.005107+0000","last_clean":"2024-03-23T07:56:22.005107+0000","last_became_active":"2024-03-23T07:56:17.725241+0000","last_became_peered":"2024-03-23T07:56:17.725241+0000","last_unstale":"2024-03-23T07:56:22.005107+0000","last_undegraded":"2024-03-23T07:56:22.005107+0000","last_fullsized":"2024-03-23T07:56:22.005107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:08:14.226813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.999373+0000","last_change":"2024-03-23T07:56:17.224220+0000","last_active":"2024-03-23T07:56:21.999373+0000","last_peered":"2024-03-23T07:56:21.999373+0000","last_clean":"2024-03-23T07:56:21.999373+0000","last_became_active":"2024-03-23T07:56:17.223476+0000","last_became_peered":"2024-03-23T07:56:17.223476+0000","last_unstale":"2024-03-23T07:56:21.999373+0000","last_undegraded":"2024-03-23T07:56:21.999373+0000","last_fullsized":"2024-03-23T07:56:21.999373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031049+0000","last_change":"2024-03-23T07:56:22.027325+0000","last_active":"2024-03-23T07:56:22.031049+0000","last_peered":"2024-03-23T07:56:22.031049+0000","last_clean":"2024-03-23T07:56:22.031049+0000","last_became_active":"2024-03-23T07:56:22.026858+0000","last_became_peered":"2024-03-23T07:56:22.026858+0000","last_unstale":"2024-03-23T07:56:22.031049+0000","last_undegraded":"2024-03-23T07:56:22.031049+0000","last_fullsized":"2024-03-23T07:56:22.031049+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:38.880449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093719+0000","last_change":"2024-03-23T07:56:17.736540+0000","last_active":"2024-03-23T07:56:22.093719+0000","last_peered":"2024-03-23T07:56:22.093719+0000","last_clean":"2024-03-23T07:56:22.093719+0000","last_became_active":"2024-03-23T07:56:17.727131+0000","last_became_peered":"2024-03-23T07:56:17.727131+0000","last_unstale":"2024-03-23T07:56:22.093719+0000","last_undegraded":"2024-03-23T07:56:22.093719+0000","last_fullsized":"2024-03-23T07:56:22.093719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.006955+0000","last_change":"2024-03-23T07:56:17.215558+0000","last_active":"2024-03-23T07:56:22.006955+0000","last_peered":"2024-03-23T07:56:22.006955+0000","last_clean":"2024-03-23T07:56:22.006955+0000","last_became_active":"2024-03-23T07:56:17.214342+0000","last_became_peered":"2024-03-23T07:56:17.214342+0000","last_unstale":"2024-03-23T07:56:22.006955+0000","last_undegraded":"2024-03-23T07:56:22.006955+0000","last_fullsized":"2024-03-23T07:56:22.006955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.010553+0000","last_change":"2024-03-23T07:56:22.008000+0000","last_active":"2024-03-23T07:56:22.010553+0000","last_peered":"2024-03-23T07:56:22.010553+0000","last_clean":"2024-03-23T07:56:22.010553+0000","last_became_active":"2024-03-23T07:56:22.006996+0000","last_became_peered":"2024-03-23T07:56:22.006996+0000","last_unstale":"2024-03-23T07:56:22.010553+0000","last_undegraded":"2024-03-23T07:56:22.010553+0000","last_fullsized":"2024-03-23T07:56:22.010553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:24:55.446700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015561+0000","last_change":"2024-03-23T07:56:22.009204+0000","last_active":"2024-03-23T07:56:22.015561+0000","last_peered":"2024-03-23T07:56:22.015561+0000","last_clean":"2024-03-23T07:56:22.015561+0000","last_became_active":"2024-03-23T07:56:22.008744+0000","last_became_peered":"2024-03-23T07:56:22.008744+0000","last_unstale":"2024-03-23T07:56:22.015561+0000","last_undegraded":"2024-03-23T07:56:22.015561+0000","last_fullsized":"2024-03-23T07:56:22.015561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:28.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984217+0000","last_change":"2024-03-23T07:56:17.733961+0000","last_active":"2024-03-23T07:56:21.984217+0000","last_peered":"2024-03-23T07:56:21.984217+0000","last_clean":"2024-03-23T07:56:21.984217+0000","last_became_active":"2024-03-23T07:56:17.733646+0000","last_became_peered":"2024-03-23T07:56:17.733646+0000","last_unstale":"2024-03-23T07:56:21.984217+0000","last_undegraded":"2024-03-23T07:56:21.984217+0000","last_fullsized":"2024-03-23T07:56:21.984217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002413+0000","last_change":"2024-03-23T07:56:17.223808+0000","last_active":"2024-03-23T07:56:22.002413+0000","last_peered":"2024-03-23T07:56:22.002413+0000","last_clean":"2024-03-23T07:56:22.002413+0000","last_became_active":"2024-03-23T07:56:17.223429+0000","last_became_peered":"2024-03-23T07:56:17.223429+0000","last_unstale":"2024-03-23T07:56:22.002413+0000","last_undegraded":"2024-03-23T07:56:22.002413+0000","last_fullsized":"2024-03-23T07:56:22.002413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:57.800903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097333+0000","last_change":"2024-03-23T07:56:22.015838+0000","last_active":"2024-03-23T07:56:22.097333+0000","last_peered":"2024-03-23T07:56:22.097333+0000","last_clean":"2024-03-23T07:56:22.097333+0000","last_became_active":"2024-03-23T07:56:22.015393+0000","last_became_peered":"2024-03-23T07:56:22.015393+0000","last_unstale":"2024-03-23T07:56:22.097333+0000","last_undegraded":"2024-03-23T07:56:22.097333+0000","last_fullsized":"2024-03-23T07:56:22.097333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:35:13.840057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985820+0000","last_change":"2024-03-23T07:56:17.729460+0000","last_active":"2024-03-23T07:56:21.985820+0000","last_peered":"2024-03-23T07:56:21.985820+0000","last_clean":"2024-03-23T07:56:21.985820+0000","last_became_active":"2024-03-23T07:56:17.728590+0000","last_became_peered":"2024-03-23T07:56:17.728590+0000","last_unstale":"2024-03-23T07:56:21.985820+0000","last_undegraded":"2024-03-23T07:56:21.985820+0000","last_fullsized":"2024-03-23T07:56:21.985820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:32:18.762479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001880+0000","last_change":"2024-03-23T07:56:17.216006+0000","last_active":"2024-03-23T07:56:22.001880+0000","last_peered":"2024-03-23T07:56:22.001880+0000","last_clean":"2024-03-23T07:56:22.001880+0000","last_became_active":"2024-03-23T07:56:17.215734+0000","last_became_peered":"2024-03-23T07:56:17.215734+0000","last_unstale":"2024-03-23T07:56:22.001880+0000","last_undegraded":"2024-03-23T07:56:22.001880+0000","last_fullsized":"2024-03-23T07:56:22.001880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003800+0000","last_change":"2024-03-23T07:56:17.225732+0000","last_active":"2024-03-23T07:56:22.003800+0000","last_peered":"2024-03-23T07:56:22.003800+0000","last_clean":"2024-03-23T07:56:22.003800+0000","last_became_active":"2024-03-23T07:56:17.225216+0000","last_became_peered":"2024-03-23T07:56:17.225216+0000","last_unstale":"2024-03-23T07:56:22.003800+0000","last_undegraded":"2024-03-23T07:56:22.003800+0000","last_fullsized":"2024-03-23T07:56:22.003800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:08.861374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982688+0000","last_change":"2024-03-23T07:56:17.750834+0000","last_active":"2024-03-23T07:56:21.982688+0000","last_peered":"2024-03-23T07:56:21.982688+0000","last_clean":"2024-03-23T07:56:21.982688+0000","last_became_active":"2024-03-23T07:56:17.750483+0000","last_became_peered":"2024-03-23T07:56:17.750483+0000","last_unstale":"2024-03-23T07:56:21.982688+0000","last_undegraded":"2024-03-23T07:56:21.982688+0000","last_fullsized":"2024-03-23T07:56:21.982688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018371+0000","last_change":"2024-03-23T07:56:22.013558+0000","last_active":"2024-03-23T07:56:22.018371+0000","last_peered":"2024-03-23T07:56:22.018371+0000","last_clean":"2024-03-23T07:56:22.018371+0000","last_became_active":"2024-03-23T07:56:22.011671+0000","last_became_peered":"2024-03-23T07:56:22.011671+0000","last_unstale":"2024-03-23T07:56:22.018371+0000","last_undegraded":"2024-03-23T07:56:22.018371+0000","last_fullsized":"2024-03-23T07:56:22.018371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:07.197610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991122+0000","last_change":"2024-03-23T07:56:17.217610+0000","last_active":"2024-03-23T07:56:21.991122+0000","last_peered":"2024-03-23T07:56:21.991122+0000","last_clean":"2024-03-23T07:56:21.991122+0000","last_became_active":"2024-03-23T07:56:17.217342+0000","last_became_peered":"2024-03-23T07:56:17.217342+0000","last_unstale":"2024-03-23T07:56:21.991122+0000","last_undegraded":"2024-03-23T07:56:21.991122+0000","last_fullsized":"2024-03-23T07:56:21.991122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:27.578107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093588+0000","last_change":"2024-03-23T07:56:17.719621+0000","last_active":"2024-03-23T07:56:22.093588+0000","last_peered":"2024-03-23T07:56:22.093588+0000","last_clean":"2024-03-23T07:56:22.093588+0000","last_became_active":"2024-03-23T07:56:17.719296+0000","last_became_peered":"2024-03-23T07:56:17.719296+0000","last_unstale":"2024-03-23T07:56:22.093588+0000","last_undegraded":"2024-03-23T07:56:22.093588+0000","last_fullsized":"2024-03-23T07:56:22.093588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031093+0000","last_change":"2024-03-23T07:56:22.027765+0000","last_active":"2024-03-23T07:56:22.031093+0000","last_peered":"2024-03-23T07:56:22.031093+0000","last_clean":"2024-03-23T07:56:22.031093+0000","last_became_active":"2024-03-23T07:56:22.027420+0000","last_became_peered":"2024-03-23T07:56:22.027420+0000","last_unstale":"2024-03-23T07:56:22.031093+0000","last_undegraded":"2024-03-23T07:56:22.031093+0000","last_fullsized":"2024-03-23T07:56:22.031093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:25.578708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093412+0000","last_change":"2024-03-23T07:56:17.721639+0000","last_active":"2024-03-23T07:56:22.093412+0000","last_peered":"2024-03-23T07:56:22.093412+0000","last_clean":"2024-03-23T07:56:22.093412+0000","last_became_active":"2024-03-23T07:56:17.721372+0000","last_became_peered":"2024-03-23T07:56:17.721372+0000","last_unstale":"2024-03-23T07:56:22.093412+0000","last_undegraded":"2024-03-23T07:56:22.093412+0000","last_fullsized":"2024-03-23T07:56:22.093412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014858+0000","last_change":"2024-03-23T07:56:17.741387+0000","last_active":"2024-03-23T07:56:22.014858+0000","last_peered":"2024-03-23T07:56:22.014858+0000","last_clean":"2024-03-23T07:56:22.014858+0000","last_became_active":"2024-03-23T07:56:17.741091+0000","last_became_peered":"2024-03-23T07:56:17.741091+0000","last_unstale":"2024-03-23T07:56:22.014858+0000","last_undegraded":"2024-03-23T07:56:22.014858+0000","last_fullsized":"2024-03-23T07:56:22.014858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097117+0000","last_change":"2024-03-23T07:56:17.729701+0000","last_active":"2024-03-23T07:56:22.097117+0000","last_peered":"2024-03-23T07:56:22.097117+0000","last_clean":"2024-03-23T07:56:22.097117+0000","last_became_active":"2024-03-23T07:56:17.729481+0000","last_became_peered":"2024-03-23T07:56:17.729481+0000","last_unstale":"2024-03-23T07:56:22.097117+0000","last_undegraded":"2024-03-23T07:56:22.097117+0000","last_fullsized":"2024-03-23T07:56:22.097117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.990887+0000","last_change":"2024-03-23T07:56:17.724914+0000","last_active":"2024-03-23T07:56:21.990887+0000","last_peered":"2024-03-23T07:56:21.990887+0000","last_clean":"2024-03-23T07:56:21.990887+0000","last_became_active":"2024-03-23T07:56:17.724627+0000","last_became_peered":"2024-03-23T07:56:17.724627+0000","last_unstale":"2024-03-23T07:56:21.990887+0000","last_undegraded":"2024-03-23T07:56:21.990887+0000","last_fullsized":"2024-03-23T07:56:21.990887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:36.852829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002999+0000","last_change":"2024-03-23T07:56:17.745053+0000","last_active":"2024-03-23T07:56:22.002999+0000","last_peered":"2024-03-23T07:56:22.002999+0000","last_clean":"2024-03-23T07:56:22.002999+0000","last_became_active":"2024-03-23T07:56:17.744775+0000","last_became_peered":"2024-03-23T07:56:17.744775+0000","last_unstale":"2024-03-23T07:56:22.002999+0000","last_undegraded":"2024-03-23T07:56:22.002999+0000","last_fullsized":"2024-03-23T07:56:22.002999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001976+0000","last_change":"2024-03-23T07:56:17.726849+0000","last_active":"2024-03-23T07:56:22.001976+0000","last_peered":"2024-03-23T07:56:22.001976+0000","last_clean":"2024-03-23T07:56:22.001976+0000","last_became_active":"2024-03-23T07:56:17.726339+0000","last_became_peered":"2024-03-23T07:56:17.726339+0000","last_unstale":"2024-03-23T07:56:22.001976+0000","last_undegraded":"2024-03-23T07:56:22.001976+0000","last_fullsized":"2024-03-23T07:56:22.001976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:19:42.241859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093701+0000","last_change":"2024-03-23T07:56:17.736674+0000","last_active":"2024-03-23T07:56:22.093701+0000","last_peered":"2024-03-23T07:56:22.093701+0000","last_clean":"2024-03-23T07:56:22.093701+0000","last_became_active":"2024-03-23T07:56:17.728841+0000","last_became_peered":"2024-03-23T07:56:17.728841+0000","last_unstale":"2024-03-23T07:56:22.093701+0000","last_undegraded":"2024-03-23T07:56:22.093701+0000","last_fullsized":"2024-03-23T07:56:22.093701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000680+0000","last_change":"2024-03-23T07:56:17.734249+0000","last_active":"2024-03-23T07:56:22.000680+0000","last_peered":"2024-03-23T07:56:22.000680+0000","last_clean":"2024-03-23T07:56:22.000680+0000","last_became_active":"2024-03-23T07:56:17.733805+0000","last_became_peered":"2024-03-23T07:56:17.733805+0000","last_unstale":"2024-03-23T07:56:22.000680+0000","last_undegraded":"2024-03-23T07:56:22.000680+0000","last_fullsized":"2024-03-23T07:56:22.000680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:47.287496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097031+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097031+0000","last_peered":"2024-03-23T07:56:22.097031+0000","last_clean":"2024-03-23T07:56:22.097031+0000","last_became_active":"2024-03-23T07:56:17.727006+0000","last_became_peered":"2024-03-23T07:56:17.727006+0000","last_unstale":"2024-03-23T07:56:22.097031+0000","last_undegraded":"2024-03-23T07:56:22.097031+0000","last_fullsized":"2024-03-23T07:56:22.097031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:54.056625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.989132+0000","last_change":"2024-03-23T07:56:17.730066+0000","last_active":"2024-03-23T07:56:21.989132+0000","last_peered":"2024-03-23T07:56:21.989132+0000","last_clean":"2024-03-23T07:56:21.989132+0000","last_became_active":"2024-03-23T07:56:17.729250+0000","last_became_peered":"2024-03-23T07:56:17.729250+0000","last_unstale":"2024-03-23T07:56:21.989132+0000","last_undegraded":"2024-03-23T07:56:21.989132+0000","last_fullsized":"2024-03-23T07:56:21.989132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:28.507099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093550+0000","last_change":"2024-03-23T07:56:17.732206+0000","last_active":"2024-03-23T07:56:22.093550+0000","last_peered":"2024-03-23T07:56:22.093550+0000","last_clean":"2024-03-23T07:56:22.093550+0000","last_became_active":"2024-03-23T07:56:17.731391+0000","last_became_peered":"2024-03-23T07:56:17.731391+0000","last_unstale":"2024-03-23T07:56:22.093550+0000","last_undegraded":"2024-03-23T07:56:22.093550+0000","last_fullsized":"2024-03-23T07:56:22.093550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982447+0000","last_change":"2024-03-23T07:56:17.727582+0000","last_active":"2024-03-23T07:56:21.982447+0000","last_peered":"2024-03-23T07:56:21.982447+0000","last_clean":"2024-03-23T07:56:21.982447+0000","last_became_active":"2024-03-23T07:56:17.727312+0000","last_became_peered":"2024-03-23T07:56:17.727312+0000","last_unstale":"2024-03-23T07:56:21.982447+0000","last_undegraded":"2024-03-23T07:56:21.982447+0000","last_fullsized":"2024-03-23T07:56:21.982447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096494+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096494+0000","last_peered":"2024-03-23T07:56:22.096494+0000","last_clean":"2024-03-23T07:56:22.096494+0000","last_became_active":"2024-03-23T07:56:17.743717+0000","last_became_peered":"2024-03-23T07:56:17.743717+0000","last_unstale":"2024-03-23T07:56:22.096494+0000","last_undegraded":"2024-03-23T07:56:22.096494+0000","last_fullsized":"2024-03-23T07:56:22.096494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:35.095859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097297+0000","last_change":"2024-03-23T07:56:17.729192+0000","last_active":"2024-03-23T07:56:22.097297+0000","last_peered":"2024-03-23T07:56:22.097297+0000","last_clean":"2024-03-23T07:56:22.097297+0000","last_became_active":"2024-03-23T07:56:17.728922+0000","last_became_peered":"2024-03-23T07:56:17.728922+0000","last_unstale":"2024-03-23T07:56:22.097297+0000","last_undegraded":"2024-03-23T07:56:22.097297+0000","last_fullsized":"2024-03-23T07:56:22.097297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017348+0000","last_change":"2024-03-23T07:56:17.731099+0000","last_active":"2024-03-23T07:56:22.017348+0000","last_peered":"2024-03-23T07:56:22.017348+0000","last_clean":"2024-03-23T07:56:22.017348+0000","last_became_active":"2024-03-23T07:56:17.730597+0000","last_became_peered":"2024-03-23T07:56:17.730597+0000","last_unstale":"2024-03-23T07:56:22.017348+0000","last_undegraded":"2024-03-23T07:56:22.017348+0000","last_fullsized":"2024-03-23T07:56:22.017348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097349+0000","last_change":"2024-03-23T07:56:17.748094+0000","last_active":"2024-03-23T07:56:22.097349+0000","last_peered":"2024-03-23T07:56:22.097349+0000","last_clean":"2024-03-23T07:56:22.097349+0000","last_became_active":"2024-03-23T07:56:17.747867+0000","last_became_peered":"2024-03-23T07:56:17.747867+0000","last_unstale":"2024-03-23T07:56:22.097349+0000","last_undegraded":"2024-03-23T07:56:22.097349+0000","last_fullsized":"2024-03-23T07:56:22.097349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:53.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097004+0000","last_change":"2024-03-23T07:56:17.731875+0000","last_active":"2024-03-23T07:56:22.097004+0000","last_peered":"2024-03-23T07:56:22.097004+0000","last_clean":"2024-03-23T07:56:22.097004+0000","last_became_active":"2024-03-23T07:56:17.731600+0000","last_became_peered":"2024-03-23T07:56:17.731600+0000","last_unstale":"2024-03-23T07:56:22.097004+0000","last_undegraded":"2024-03-23T07:56:22.097004+0000","last_fullsized":"2024-03-23T07:56:22.097004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:55.441592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988753+0000","last_change":"2024-03-23T07:56:17.724024+0000","last_active":"2024-03-23T07:56:21.988753+0000","last_peered":"2024-03-23T07:56:21.988753+0000","last_clean":"2024-03-23T07:56:21.988753+0000","last_became_active":"2024-03-23T07:56:17.723743+0000","last_became_peered":"2024-03-23T07:56:17.723743+0000","last_unstale":"2024-03-23T07:56:21.988753+0000","last_undegraded":"2024-03-23T07:56:21.988753+0000","last_fullsized":"2024-03-23T07:56:21.988753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:32:51.077761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002205+0000","last_change":"2024-03-23T07:56:17.723640+0000","last_active":"2024-03-23T07:56:22.002205+0000","last_peered":"2024-03-23T07:56:22.002205+0000","last_clean":"2024-03-23T07:56:22.002205+0000","last_became_active":"2024-03-23T07:56:17.723352+0000","last_became_peered":"2024-03-23T07:56:17.723352+0000","last_unstale":"2024-03-23T07:56:22.002205+0000","last_undegraded":"2024-03-23T07:56:22.002205+0000","last_fullsized":"2024-03-23T07:56:22.002205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:48.788785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002849+0000","last_change":"2024-03-23T07:56:17.745108+0000","last_active":"2024-03-23T07:56:22.002849+0000","last_peered":"2024-03-23T07:56:22.002849+0000","last_clean":"2024-03-23T07:56:22.002849+0000","last_became_active":"2024-03-23T07:56:17.744769+0000","last_became_peered":"2024-03-23T07:56:17.744769+0000","last_unstale":"2024-03-23T07:56:22.002849+0000","last_undegraded":"2024-03-23T07:56:22.002849+0000","last_fullsized":"2024-03-23T07:56:22.002849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:45.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096478+0000","last_change":"2024-03-23T07:56:17.743989+0000","last_active":"2024-03-23T07:56:22.096478+0000","last_peered":"2024-03-23T07:56:22.096478+0000","last_clean":"2024-03-23T07:56:22.096478+0000","last_became_active":"2024-03-23T07:56:17.743697+0000","last_became_peered":"2024-03-23T07:56:17.743697+0000","last_unstale":"2024-03-23T07:56:22.096478+0000","last_undegraded":"2024-03-23T07:56:22.096478+0000","last_fullsized":"2024-03-23T07:56:22.096478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:33:48.226604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096538+0000","last_change":"2024-03-23T07:56:17.737898+0000","last_active":"2024-03-23T07:56:22.096538+0000","last_peered":"2024-03-23T07:56:22.096538+0000","last_clean":"2024-03-23T07:56:22.096538+0000","last_became_active":"2024-03-23T07:56:17.737650+0000","last_became_peered":"2024-03-23T07:56:17.737650+0000","last_unstale":"2024-03-23T07:56:22.096538+0000","last_undegraded":"2024-03-23T07:56:22.096538+0000","last_fullsized":"2024-03-23T07:56:22.096538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:21.794118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096613+0000","last_change":"2024-03-23T07:56:17.741344+0000","last_active":"2024-03-23T07:56:22.096613+0000","last_peered":"2024-03-23T07:56:22.096613+0000","last_clean":"2024-03-23T07:56:22.096613+0000","last_became_active":"2024-03-23T07:56:17.740336+0000","last_became_peered":"2024-03-23T07:56:17.740336+0000","last_unstale":"2024-03-23T07:56:22.096613+0000","last_undegraded":"2024-03-23T07:56:22.096613+0000","last_fullsized":"2024-03-23T07:56:22.096613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093627+0000","last_change":"2024-03-23T07:56:17.725091+0000","last_active":"2024-03-23T07:56:22.093627+0000","last_peered":"2024-03-23T07:56:22.093627+0000","last_clean":"2024-03-23T07:56:22.093627+0000","last_became_active":"2024-03-23T07:56:17.724794+0000","last_became_peered":"2024-03-23T07:56:17.724794+0000","last_unstale":"2024-03-23T07:56:22.093627+0000","last_undegraded":"2024-03-23T07:56:22.093627+0000","last_fullsized":"2024-03-23T07:56:22.093627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983064+0000","last_change":"2024-03-23T07:56:17.749204+0000","last_active":"2024-03-23T07:56:21.983064+0000","last_peered":"2024-03-23T07:56:21.983064+0000","last_clean":"2024-03-23T07:56:21.983064+0000","last_became_active":"2024-03-23T07:56:17.748888+0000","last_became_peered":"2024-03-23T07:56:17.748888+0000","last_unstale":"2024-03-23T07:56:21.983064+0000","last_undegraded":"2024-03-23T07:56:21.983064+0000","last_fullsized":"2024-03-23T07:56:21.983064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:14.142988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096378+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096378+0000","last_peered":"2024-03-23T07:56:22.096378+0000","last_clean":"2024-03-23T07:56:22.096378+0000","last_became_active":"2024-03-23T07:56:17.743764+0000","last_became_peered":"2024-03-23T07:56:17.743764+0000","last_unstale":"2024-03-23T07:56:22.096378+0000","last_undegraded":"2024-03-23T07:56:22.096378+0000","last_fullsized":"2024-03-23T07:56:22.096378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093513+0000","last_change":"2024-03-23T07:56:17.724709+0000","last_active":"2024-03-23T07:56:22.093513+0000","last_peered":"2024-03-23T07:56:22.093513+0000","last_clean":"2024-03-23T07:56:22.093513+0000","last_became_active":"2024-03-23T07:56:17.724393+0000","last_became_peered":"2024-03-23T07:56:17.724393+0000","last_unstale":"2024-03-23T07:56:22.093513+0000","last_undegraded":"2024-03-23T07:56:22.093513+0000","last_fullsized":"2024-03-23T07:56:22.093513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096409+0000","last_change":"2024-03-23T07:56:17.742970+0000","last_active":"2024-03-23T07:56:22.096409+0000","last_peered":"2024-03-23T07:56:22.096409+0000","last_clean":"2024-03-23T07:56:22.096409+0000","last_became_active":"2024-03-23T07:56:17.742629+0000","last_became_peered":"2024-03-23T07:56:17.742629+0000","last_unstale":"2024-03-23T07:56:22.096409+0000","last_undegraded":"2024-03-23T07:56:22.096409+0000","last_fullsized":"2024-03-23T07:56:22.096409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:21:07.705635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093633+0000","last_change":"2024-03-23T07:56:17.736446+0000","last_active":"2024-03-23T07:56:22.093633+0000","last_peered":"2024-03-23T07:56:22.093633+0000","last_clean":"2024-03-23T07:56:22.093633+0000","last_became_active":"2024-03-23T07:56:17.727700+0000","last_became_peered":"2024-03-23T07:56:17.727700+0000","last_unstale":"2024-03-23T07:56:22.093633+0000","last_undegraded":"2024-03-23T07:56:22.093633+0000","last_fullsized":"2024-03-23T07:56:22.093633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991994+0000","last_change":"2024-03-23T07:56:17.732549+0000","last_active":"2024-03-23T07:56:21.991994+0000","last_peered":"2024-03-23T07:56:21.991994+0000","last_clean":"2024-03-23T07:56:21.991994+0000","last_became_active":"2024-03-23T07:56:17.732276+0000","last_became_peered":"2024-03-23T07:56:17.732276+0000","last_unstale":"2024-03-23T07:56:21.991994+0000","last_undegraded":"2024-03-23T07:56:21.991994+0000","last_fullsized":"2024-03-23T07:56:21.991994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:48.012231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000782+0000","last_change":"2024-03-23T07:56:17.732034+0000","last_active":"2024-03-23T07:56:22.000782+0000","last_peered":"2024-03-23T07:56:22.000782+0000","last_clean":"2024-03-23T07:56:22.000782+0000","last_became_active":"2024-03-23T07:56:17.731730+0000","last_became_peered":"2024-03-23T07:56:17.731730+0000","last_unstale":"2024-03-23T07:56:22.000782+0000","last_undegraded":"2024-03-23T07:56:22.000782+0000","last_fullsized":"2024-03-23T07:56:22.000782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:59:20.863874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096639+0000","last_change":"2024-03-23T07:56:17.746986+0000","last_active":"2024-03-23T07:56:22.096639+0000","last_peered":"2024-03-23T07:56:22.096639+0000","last_clean":"2024-03-23T07:56:22.096639+0000","last_became_active":"2024-03-23T07:56:17.746621+0000","last_became_peered":"2024-03-23T07:56:17.746621+0000","last_unstale":"2024-03-23T07:56:22.096639+0000","last_undegraded":"2024-03-23T07:56:22.096639+0000","last_fullsized":"2024-03-23T07:56:22.096639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:22.719041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476745,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]}]},{"osd":2,"up_from":16,"seq":68719476745,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]}]},{"osd":1,"up_from":16,"seq":68719476745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":694448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]}]},{"osd":0,"up_from":16,"seq":68719476747,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.871}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6599999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":108477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:56:46.557 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:56:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.803+0000 7f155ae00640 1 Processor -- start 2024-03-23T07:56:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.803+0000 7f155ae00640 1 -- start start 2024-03-23T07:56:46.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f155ae00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f15540adbc0 0x7f15540b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f155ae00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15540cbab0 0x7f15540b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f155ae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155406d390 0x7f155406d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f155ae00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155406dd60 con 0x7f15540cbab0 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f155ae00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15540b16a0 con 0x7f15540adbc0 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f155ae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15540b4700 con 0x7f155406d390 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f1553400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15540cbab0 0x7f15540b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f1553400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15540cbab0 0x7f15540b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46572/0 (socket says 172.21.15.105:46572) 2024-03-23T07:56:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f1553400640 1 -- 172.21.15.105:0/950148946 learned_addr learned my addr 172.21.15.105:0/950148946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f1553e00640 1 --2- 172.21.15.105:0/950148946 >> v2:172.21.15.132:3300/0 conn(0x7f15540adbc0 0x7f15540b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f1553400640 1 -- 172.21.15.105:0/950148946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155406d390 msgr2=0x7f155406d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.805+0000 7f1558e00640 1 --2- 172.21.15.105:0/950148946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155406d390 0x7f155406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.806+0000 7f1553400640 1 --2- 172.21.15.105:0/950148946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155406d390 0x7f155406d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.806+0000 7f1553400640 1 -- 172.21.15.105:0/950148946 >> v2:172.21.15.132:3300/0 conn(0x7f15540adbc0 msgr2=0x7f15540b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.806+0000 7f1553400640 1 --2- 172.21.15.105:0/950148946 >> v2:172.21.15.132:3300/0 conn(0x7f15540adbc0 0x7f15540b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.806+0000 7f1553400640 1 -- 172.21.15.105:0/950148946 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540cbfb0 con 0x7f15540cbab0 2024-03-23T07:56:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.806+0000 7f1553e00640 1 --2- 172.21.15.105:0/950148946 >> v2:172.21.15.132:3300/0 conn(0x7f15540adbc0 0x7f15540b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:46.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.807+0000 7f1553400640 1 --2- 172.21.15.105:0/950148946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15540cbab0 0x7f15540b20f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1540009870 tx=0x7f154002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f712fa9e2568e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.807+0000 7f1552a00640 1 -- 172.21.15.105:0/950148946 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400317c0 con 0x7f15540cbab0 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.807+0000 7f1552a00640 1 -- 172.21.15.105:0/950148946 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1540031920 con 0x7f15540cbab0 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.807+0000 7f1552a00640 1 -- 172.21.15.105:0/950148946 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540031cb0 con 0x7f15540cbab0 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.807+0000 7f155ae00640 1 -- 172.21.15.105:0/950148946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15540cbab0 msgr2=0x7f15540b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.807+0000 7f155ae00640 1 --2- 172.21.15.105:0/950148946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15540cbab0 0x7f15540b20f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1540009870 tx=0x7f154002b860 comp rx=0 tx=0).stop 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.808+0000 7f155ae00640 1 -- 172.21.15.105:0/950148946 shutdown_connections 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.808+0000 7f155ae00640 1 --2- 172.21.15.105:0/950148946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155406d390 0x7f155406d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.808+0000 7f155ae00640 1 --2- 172.21.15.105:0/950148946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15540cbab0 0x7f15540b20f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.808+0000 7f155ae00640 1 --2- 172.21.15.105:0/950148946 >> v2:172.21.15.132:3300/0 conn(0x7f15540adbc0 0x7f15540b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.808+0000 7f155ae00640 1 -- 172.21.15.105:0/950148946 >> 172.21.15.105:0/950148946 conn(0x7f155406a3d0 msgr2=0x7f15540b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.808+0000 7f155ae00640 1 -- 172.21.15.105:0/950148946 shutdown_connections 2024-03-23T07:56:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.808+0000 7f155ae00640 1 -- 172.21.15.105:0/950148946 wait complete. 2024-03-23T07:56:46.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.809+0000 7f155ae00640 1 Processor -- start 2024-03-23T07:56:46.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.810+0000 7f155ae00640 1 -- start start 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.810+0000 7f155ae00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155406d390 0x7f155413dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.810+0000 7f155ae00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15540adbc0 0x7f155413fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.810+0000 7f155ae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15540cbab0 0x7f155415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.810+0000 7f155ae00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15541411f0 con 0x7f155406d390 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.810+0000 7f155ae00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554140570 con 0x7f15540adbc0 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.810+0000 7f155ae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1554140850 con 0x7f15540cbab0 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155406d390 0x7f155413dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155406d390 0x7f155413dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46588/0 (socket says 172.21.15.105:46588) 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 -- 172.21.15.105:0/1974714314 learned_addr learned my addr 172.21.15.105:0/1974714314 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 -- 172.21.15.105:0/1974714314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15540cbab0 msgr2=0x7f155415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15540cbab0 0x7f155415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 -- 172.21.15.105:0/1974714314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15540adbc0 msgr2=0x7f155413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15540adbc0 0x7f155413fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 -- 172.21.15.105:0/1974714314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15400094d0 con 0x7f155406d390 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.811+0000 7f1553e00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155406d390 0x7f155413dd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f154400c7f0 tx=0x7f154400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.812+0000 7f154be00640 1 -- 172.21.15.105:0/1974714314 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544007a30 con 0x7f155406d390 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.812+0000 7f154be00640 1 -- 172.21.15.105:0/1974714314 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1544007b90 con 0x7f155406d390 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.812+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1554140b30 con 0x7f155406d390 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.812+0000 7f154be00640 1 -- 172.21.15.105:0/1974714314 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154400ce40 con 0x7f155406d390 2024-03-23T07:56:46.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.812+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1554141610 con 0x7f155406d390 2024-03-23T07:56:46.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.813+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155406ac20 con 0x7f155406d390 2024-03-23T07:56:46.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.814+0000 7f154be00640 1 -- 172.21.15.105:0/1974714314 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f15440113f0 con 0x7f155406d390 2024-03-23T07:56:46.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.815+0000 7f154be00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f153406ac30 0x7f153406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:46.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.815+0000 7f154be00640 1 -- 172.21.15.105:0/1974714314 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1544089940 con 0x7f155406d390 2024-03-23T07:56:46.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.815+0000 7f1553400640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f153406ac30 0x7f153406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:46.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.815+0000 7f1553400640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f153406ac30 0x7f153406d0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1540036000 tx=0x7f1540030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:46.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:46.819+0000 7f154be00640 1 -- 172.21.15.105:0/1974714314 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1544054710 con 0x7f155406d390 2024-03-23T07:56:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.009+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15540b18f0 con 0x7f153406ac30 2024-03-23T07:56:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.019+0000 7f154be00640 1 -- 172.21.15.105:0/1974714314 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345105 (secure 0 0 0) 0x7f1540031b10 con 0x7f153406ac30 2024-03-23T07:56:47.025 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:47.026 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:56:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.025+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f153406ac30 msgr2=0x7f153406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.025+0000 7f155ae00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f153406ac30 0x7f153406d0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1540036000 tx=0x7f1540030000 comp rx=0 tx=0).stop 2024-03-23T07:56:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.025+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155406d390 msgr2=0x7f155413dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.025+0000 7f155ae00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155406d390 0x7f155413dd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f154400c7f0 tx=0x7f154400cbf0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.025+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 shutdown_connections 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.026+0000 7f155ae00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f153406ac30 0x7f153406d0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.026+0000 7f155ae00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15540cbab0 0x7f155415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.026+0000 7f155ae00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15540adbc0 0x7f155413fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.026+0000 7f155ae00640 1 --2- 172.21.15.105:0/1974714314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155406d390 0x7f155413dd50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.026+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 >> 172.21.15.105:0/1974714314 conn(0x7f155406a3d0 msgr2=0x7f15540b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.026+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 shutdown_connections 2024-03-23T07:56:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.026+0000 7f155ae00640 1 -- 172.21.15.105:0/1974714314 wait complete. 2024-03-23T07:56:47.045 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-23T07:56:46.050598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2022002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004962"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995336+0000","last_change":"2024-03-23T07:56:17.227981+0000","last_active":"2024-03-23T07:56:21.995336+0000","last_peered":"2024-03-23T07:56:21.995336+0000","last_clean":"2024-03-23T07:56:21.995336+0000","last_became_active":"2024-03-23T07:56:17.227576+0000","last_became_peered":"2024-03-23T07:56:17.227576+0000","last_unstale":"2024-03-23T07:56:21.995336+0000","last_undegraded":"2024-03-23T07:56:21.995336+0000","last_fullsized":"2024-03-23T07:56:21.995336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001380+0000","last_change":"2024-03-23T07:56:17.730054+0000","last_active":"2024-03-23T07:56:22.001380+0000","last_peered":"2024-03-23T07:56:22.001380+0000","last_clean":"2024-03-23T07:56:22.001380+0000","last_became_active":"2024-03-23T07:56:17.729766+0000","last_became_peered":"2024-03-23T07:56:17.729766+0000","last_unstale":"2024-03-23T07:56:22.001380+0000","last_undegraded":"2024-03-23T07:56:22.001380+0000","last_fullsized":"2024-03-23T07:56:22.001380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:39.987411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.032954+0000","last_change":"2024-03-23T07:56:22.029375+0000","last_active":"2024-03-23T07:56:22.032954+0000","last_peered":"2024-03-23T07:56:22.032954+0000","last_clean":"2024-03-23T07:56:22.032954+0000","last_became_active":"2024-03-23T07:56:22.028383+0000","last_became_peered":"2024-03-23T07:56:22.028383+0000","last_unstale":"2024-03-23T07:56:22.032954+0000","last_undegraded":"2024-03-23T07:56:22.032954+0000","last_fullsized":"2024-03-23T07:56:22.032954+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:06.760193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001542+0000","last_change":"2024-03-23T07:56:17.217630+0000","last_active":"2024-03-23T07:56:22.001542+0000","last_peered":"2024-03-23T07:56:22.001542+0000","last_clean":"2024-03-23T07:56:22.001542+0000","last_became_active":"2024-03-23T07:56:17.217179+0000","last_became_peered":"2024-03-23T07:56:17.217179+0000","last_unstale":"2024-03-23T07:56:22.001542+0000","last_undegraded":"2024-03-23T07:56:22.001542+0000","last_fullsized":"2024-03-23T07:56:22.001542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:27:59.587867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004329+0000","last_change":"2024-03-23T07:56:17.734047+0000","last_active":"2024-03-23T07:56:22.004329+0000","last_peered":"2024-03-23T07:56:22.004329+0000","last_clean":"2024-03-23T07:56:22.004329+0000","last_became_active":"2024-03-23T07:56:17.733482+0000","last_became_peered":"2024-03-23T07:56:17.733482+0000","last_unstale":"2024-03-23T07:56:22.004329+0000","last_undegraded":"2024-03-23T07:56:22.004329+0000","last_fullsized":"2024-03-23T07:56:22.004329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:47:35.800634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093379+0000","last_change":"2024-03-23T07:56:22.012905+0000","last_active":"2024-03-23T07:56:22.093379+0000","last_peered":"2024-03-23T07:56:22.093379+0000","last_clean":"2024-03-23T07:56:22.093379+0000","last_became_active":"2024-03-23T07:56:22.012340+0000","last_became_peered":"2024-03-23T07:56:22.012340+0000","last_unstale":"2024-03-23T07:56:22.093379+0000","last_undegraded":"2024-03-23T07:56:22.093379+0000","last_fullsized":"2024-03-23T07:56:22.093379+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:43.862034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000002+0000","last_change":"2024-03-23T07:56:17.224614+0000","last_active":"2024-03-23T07:56:22.000002+0000","last_peered":"2024-03-23T07:56:22.000002+0000","last_clean":"2024-03-23T07:56:22.000002+0000","last_became_active":"2024-03-23T07:56:17.223463+0000","last_became_peered":"2024-03-23T07:56:17.223463+0000","last_unstale":"2024-03-23T07:56:22.000002+0000","last_undegraded":"2024-03-23T07:56:22.000002+0000","last_fullsized":"2024-03-23T07:56:22.000002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093607+0000","last_change":"2024-03-23T07:56:17.729551+0000","last_active":"2024-03-23T07:56:22.093607+0000","last_peered":"2024-03-23T07:56:22.093607+0000","last_clean":"2024-03-23T07:56:22.093607+0000","last_became_active":"2024-03-23T07:56:17.729295+0000","last_became_peered":"2024-03-23T07:56:17.729295+0000","last_unstale":"2024-03-23T07:56:22.093607+0000","last_undegraded":"2024-03-23T07:56:22.093607+0000","last_fullsized":"2024-03-23T07:56:22.093607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093533+0000","last_change":"2024-03-23T07:56:22.005574+0000","last_active":"2024-03-23T07:56:22.093533+0000","last_peered":"2024-03-23T07:56:22.093533+0000","last_clean":"2024-03-23T07:56:22.093533+0000","last_became_active":"2024-03-23T07:56:22.005169+0000","last_became_peered":"2024-03-23T07:56:22.005169+0000","last_unstale":"2024-03-23T07:56:22.093533+0000","last_undegraded":"2024-03-23T07:56:22.093533+0000","last_fullsized":"2024-03-23T07:56:22.093533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005504+0000","last_change":"2024-03-23T07:56:17.223235+0000","last_active":"2024-03-23T07:56:22.005504+0000","last_peered":"2024-03-23T07:56:22.005504+0000","last_clean":"2024-03-23T07:56:22.005504+0000","last_became_active":"2024-03-23T07:56:17.222925+0000","last_became_peered":"2024-03-23T07:56:17.222925+0000","last_unstale":"2024-03-23T07:56:22.005504+0000","last_undegraded":"2024-03-23T07:56:22.005504+0000","last_fullsized":"2024-03-23T07:56:22.005504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097254+0000","last_change":"2024-03-23T07:56:17.745979+0000","last_active":"2024-03-23T07:56:22.097254+0000","last_peered":"2024-03-23T07:56:22.097254+0000","last_clean":"2024-03-23T07:56:22.097254+0000","last_became_active":"2024-03-23T07:56:17.745702+0000","last_became_peered":"2024-03-23T07:56:17.745702+0000","last_unstale":"2024-03-23T07:56:22.097254+0000","last_undegraded":"2024-03-23T07:56:22.097254+0000","last_fullsized":"2024-03-23T07:56:22.097254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:07.760652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093472+0000","last_change":"2024-03-23T07:56:22.021092+0000","last_active":"2024-03-23T07:56:22.093472+0000","last_peered":"2024-03-23T07:56:22.093472+0000","last_clean":"2024-03-23T07:56:22.093472+0000","last_became_active":"2024-03-23T07:56:22.017901+0000","last_became_peered":"2024-03-23T07:56:22.017901+0000","last_unstale":"2024-03-23T07:56:22.093472+0000","last_undegraded":"2024-03-23T07:56:22.093472+0000","last_fullsized":"2024-03-23T07:56:22.093472+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097129+0000","last_change":"2024-03-23T07:56:17.733964+0000","last_active":"2024-03-23T07:56:22.097129+0000","last_peered":"2024-03-23T07:56:22.097129+0000","last_clean":"2024-03-23T07:56:22.097129+0000","last_became_active":"2024-03-23T07:56:17.733721+0000","last_became_peered":"2024-03-23T07:56:17.733721+0000","last_unstale":"2024-03-23T07:56:22.097129+0000","last_undegraded":"2024-03-23T07:56:22.097129+0000","last_fullsized":"2024-03-23T07:56:22.097129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004014+0000","last_change":"2024-03-23T07:56:17.227903+0000","last_active":"2024-03-23T07:56:22.004014+0000","last_peered":"2024-03-23T07:56:22.004014+0000","last_clean":"2024-03-23T07:56:22.004014+0000","last_became_active":"2024-03-23T07:56:17.227224+0000","last_became_peered":"2024-03-23T07:56:17.227224+0000","last_unstale":"2024-03-23T07:56:22.004014+0000","last_undegraded":"2024-03-23T07:56:22.004014+0000","last_fullsized":"2024-03-23T07:56:22.004014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097221+0000","last_change":"2024-03-23T07:56:22.023790+0000","last_active":"2024-03-23T07:56:22.097221+0000","last_peered":"2024-03-23T07:56:22.097221+0000","last_clean":"2024-03-23T07:56:22.097221+0000","last_became_active":"2024-03-23T07:56:22.022870+0000","last_became_peered":"2024-03-23T07:56:22.022870+0000","last_unstale":"2024-03-23T07:56:22.097221+0000","last_undegraded":"2024-03-23T07:56:22.097221+0000","last_fullsized":"2024-03-23T07:56:22.097221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:40.283526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014342+0000","last_change":"2024-03-23T07:56:17.734095+0000","last_active":"2024-03-23T07:56:22.014342+0000","last_peered":"2024-03-23T07:56:22.014342+0000","last_clean":"2024-03-23T07:56:22.014342+0000","last_became_active":"2024-03-23T07:56:17.733771+0000","last_became_peered":"2024-03-23T07:56:17.733771+0000","last_unstale":"2024-03-23T07:56:22.014342+0000","last_undegraded":"2024-03-23T07:56:22.014342+0000","last_fullsized":"2024-03-23T07:56:22.014342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002387+0000","last_change":"2024-03-23T07:56:17.221331+0000","last_active":"2024-03-23T07:56:22.002387+0000","last_peered":"2024-03-23T07:56:22.002387+0000","last_clean":"2024-03-23T07:56:22.002387+0000","last_became_active":"2024-03-23T07:56:17.221062+0000","last_became_peered":"2024-03-23T07:56:17.221062+0000","last_unstale":"2024-03-23T07:56:22.002387+0000","last_undegraded":"2024-03-23T07:56:22.002387+0000","last_fullsized":"2024-03-23T07:56:22.002387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096460+0000","last_change":"2024-03-23T07:56:22.028190+0000","last_active":"2024-03-23T07:56:22.096460+0000","last_peered":"2024-03-23T07:56:22.096460+0000","last_clean":"2024-03-23T07:56:22.096460+0000","last_became_active":"2024-03-23T07:56:22.027188+0000","last_became_peered":"2024-03-23T07:56:22.027188+0000","last_unstale":"2024-03-23T07:56:22.096460+0000","last_undegraded":"2024-03-23T07:56:22.096460+0000","last_fullsized":"2024-03-23T07:56:22.096460+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985338+0000","last_change":"2024-03-23T07:56:17.728823+0000","last_active":"2024-03-23T07:56:21.985338+0000","last_peered":"2024-03-23T07:56:21.985338+0000","last_clean":"2024-03-23T07:56:21.985338+0000","last_became_active":"2024-03-23T07:56:17.728504+0000","last_became_peered":"2024-03-23T07:56:17.728504+0000","last_unstale":"2024-03-23T07:56:21.985338+0000","last_undegraded":"2024-03-23T07:56:21.985338+0000","last_fullsized":"2024-03-23T07:56:21.985338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:28.882493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984446+0000","last_change":"2024-03-23T07:56:17.209430+0000","last_active":"2024-03-23T07:56:21.984446+0000","last_peered":"2024-03-23T07:56:21.984446+0000","last_clean":"2024-03-23T07:56:21.984446+0000","last_became_active":"2024-03-23T07:56:17.207873+0000","last_became_peered":"2024-03-23T07:56:17.207873+0000","last_unstale":"2024-03-23T07:56:21.984446+0000","last_undegraded":"2024-03-23T07:56:21.984446+0000","last_fullsized":"2024-03-23T07:56:21.984446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.033429+0000","last_change":"2024-03-23T07:56:22.031440+0000","last_active":"2024-03-23T07:56:22.033429+0000","last_peered":"2024-03-23T07:56:22.033429+0000","last_clean":"2024-03-23T07:56:22.033429+0000","last_became_active":"2024-03-23T07:56:22.030403+0000","last_became_peered":"2024-03-23T07:56:22.030403+0000","last_unstale":"2024-03-23T07:56:22.033429+0000","last_undegraded":"2024-03-23T07:56:22.033429+0000","last_fullsized":"2024-03-23T07:56:22.033429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:40.289743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097084+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097084+0000","last_peered":"2024-03-23T07:56:22.097084+0000","last_clean":"2024-03-23T07:56:22.097084+0000","last_became_active":"2024-03-23T07:56:17.728164+0000","last_became_peered":"2024-03-23T07:56:17.728164+0000","last_unstale":"2024-03-23T07:56:22.097084+0000","last_undegraded":"2024-03-23T07:56:22.097084+0000","last_fullsized":"2024-03-23T07:56:22.097084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:11:45.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998174+0000","last_change":"2024-03-23T07:56:17.211209+0000","last_active":"2024-03-23T07:56:21.998174+0000","last_peered":"2024-03-23T07:56:21.998174+0000","last_clean":"2024-03-23T07:56:21.998174+0000","last_became_active":"2024-03-23T07:56:17.210792+0000","last_became_peered":"2024-03-23T07:56:17.210792+0000","last_unstale":"2024-03-23T07:56:21.998174+0000","last_undegraded":"2024-03-23T07:56:21.998174+0000","last_fullsized":"2024-03-23T07:56:21.998174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093682+0000","last_change":"2024-03-23T07:56:22.016254+0000","last_active":"2024-03-23T07:56:22.093682+0000","last_peered":"2024-03-23T07:56:22.093682+0000","last_clean":"2024-03-23T07:56:22.093682+0000","last_became_active":"2024-03-23T07:56:22.015890+0000","last_became_peered":"2024-03-23T07:56:22.015890+0000","last_unstale":"2024-03-23T07:56:22.093682+0000","last_undegraded":"2024-03-23T07:56:22.093682+0000","last_fullsized":"2024-03-23T07:56:22.093682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:15.477490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992721+0000","last_change":"2024-03-23T07:56:17.726007+0000","last_active":"2024-03-23T07:56:21.992721+0000","last_peered":"2024-03-23T07:56:21.992721+0000","last_clean":"2024-03-23T07:56:21.992721+0000","last_became_active":"2024-03-23T07:56:17.723965+0000","last_became_peered":"2024-03-23T07:56:17.723965+0000","last_unstale":"2024-03-23T07:56:21.992721+0000","last_undegraded":"2024-03-23T07:56:21.992721+0000","last_fullsized":"2024-03-23T07:56:21.992721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004697+0000","last_change":"2024-03-23T07:56:17.225692+0000","last_active":"2024-03-23T07:56:22.004697+0000","last_peered":"2024-03-23T07:56:22.004697+0000","last_clean":"2024-03-23T07:56:22.004697+0000","last_became_active":"2024-03-23T07:56:17.225384+0000","last_became_peered":"2024-03-23T07:56:17.225384+0000","last_unstale":"2024-03-23T07:56:22.004697+0000","last_undegraded":"2024-03-23T07:56:22.004697+0000","last_fullsized":"2024-03-23T07:56:22.004697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:21.655121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093432+0000","last_change":"2024-03-23T07:56:22.007586+0000","last_active":"2024-03-23T07:56:22.093432+0000","last_peered":"2024-03-23T07:56:22.093432+0000","last_clean":"2024-03-23T07:56:22.093432+0000","last_became_active":"2024-03-23T07:56:22.007159+0000","last_became_peered":"2024-03-23T07:56:22.007159+0000","last_unstale":"2024-03-23T07:56:22.093432+0000","last_undegraded":"2024-03-23T07:56:22.093432+0000","last_fullsized":"2024-03-23T07:56:22.093432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995105+0000","last_change":"2024-03-23T07:56:17.722742+0000","last_active":"2024-03-23T07:56:21.995105+0000","last_peered":"2024-03-23T07:56:21.995105+0000","last_clean":"2024-03-23T07:56:21.995105+0000","last_became_active":"2024-03-23T07:56:17.722246+0000","last_became_peered":"2024-03-23T07:56:17.722246+0000","last_unstale":"2024-03-23T07:56:21.995105+0000","last_undegraded":"2024-03-23T07:56:21.995105+0000","last_fullsized":"2024-03-23T07:56:21.995105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994508+0000","last_change":"2024-03-23T07:56:17.212269+0000","last_active":"2024-03-23T07:56:21.994508+0000","last_peered":"2024-03-23T07:56:21.994508+0000","last_clean":"2024-03-23T07:56:21.994508+0000","last_became_active":"2024-03-23T07:56:17.211965+0000","last_became_peered":"2024-03-23T07:56:17.211965+0000","last_unstale":"2024-03-23T07:56:21.994508+0000","last_undegraded":"2024-03-23T07:56:21.994508+0000","last_fullsized":"2024-03-23T07:56:21.994508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015727+0000","last_change":"2024-03-23T07:56:22.009793+0000","last_active":"2024-03-23T07:56:22.015727+0000","last_peered":"2024-03-23T07:56:22.015727+0000","last_clean":"2024-03-23T07:56:22.015727+0000","last_became_active":"2024-03-23T07:56:22.009417+0000","last_became_peered":"2024-03-23T07:56:22.009417+0000","last_unstale":"2024-03-23T07:56:22.015727+0000","last_undegraded":"2024-03-23T07:56:22.015727+0000","last_fullsized":"2024-03-23T07:56:22.015727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003091+0000","last_change":"2024-03-23T07:56:17.726582+0000","last_active":"2024-03-23T07:56:22.003091+0000","last_peered":"2024-03-23T07:56:22.003091+0000","last_clean":"2024-03-23T07:56:22.003091+0000","last_became_active":"2024-03-23T07:56:17.726338+0000","last_became_peered":"2024-03-23T07:56:17.726338+0000","last_unstale":"2024-03-23T07:56:22.003091+0000","last_undegraded":"2024-03-23T07:56:22.003091+0000","last_fullsized":"2024-03-23T07:56:22.003091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004594+0000","last_change":"2024-03-23T07:56:17.215985+0000","last_active":"2024-03-23T07:56:22.004594+0000","last_peered":"2024-03-23T07:56:22.004594+0000","last_clean":"2024-03-23T07:56:22.004594+0000","last_became_active":"2024-03-23T07:56:17.215709+0000","last_became_peered":"2024-03-23T07:56:17.215709+0000","last_unstale":"2024-03-23T07:56:22.004594+0000","last_undegraded":"2024-03-23T07:56:22.004594+0000","last_fullsized":"2024-03-23T07:56:22.004594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028361+0000","last_change":"2024-03-23T07:56:22.026672+0000","last_active":"2024-03-23T07:56:22.028361+0000","last_peered":"2024-03-23T07:56:22.028361+0000","last_clean":"2024-03-23T07:56:22.028361+0000","last_became_active":"2024-03-23T07:56:22.025340+0000","last_became_peered":"2024-03-23T07:56:22.025340+0000","last_unstale":"2024-03-23T07:56:22.028361+0000","last_undegraded":"2024-03-23T07:56:22.028361+0000","last_fullsized":"2024-03-23T07:56:22.028361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:59.226182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093663+0000","last_change":"2024-03-23T07:56:17.723057+0000","last_active":"2024-03-23T07:56:22.093663+0000","last_peered":"2024-03-23T07:56:22.093663+0000","last_clean":"2024-03-23T07:56:22.093663+0000","last_became_active":"2024-03-23T07:56:17.722787+0000","last_became_peered":"2024-03-23T07:56:17.722787+0000","last_unstale":"2024-03-23T07:56:22.093663+0000","last_undegraded":"2024-03-23T07:56:22.093663+0000","last_fullsized":"2024-03-23T07:56:22.093663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992051+0000","last_change":"2024-03-23T07:56:17.217582+0000","last_active":"2024-03-23T07:56:21.992051+0000","last_peered":"2024-03-23T07:56:21.992051+0000","last_clean":"2024-03-23T07:56:21.992051+0000","last_became_active":"2024-03-23T07:56:17.217205+0000","last_became_peered":"2024-03-23T07:56:17.217205+0000","last_unstale":"2024-03-23T07:56:21.992051+0000","last_undegraded":"2024-03-23T07:56:21.992051+0000","last_fullsized":"2024-03-23T07:56:21.992051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:23:32.182551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.027442+0000","last_change":"2024-03-23T07:56:22.023281+0000","last_active":"2024-03-23T07:56:22.027442+0000","last_peered":"2024-03-23T07:56:22.027442+0000","last_clean":"2024-03-23T07:56:22.027442+0000","last_became_active":"2024-03-23T07:56:22.022756+0000","last_became_peered":"2024-03-23T07:56:22.022756+0000","last_unstale":"2024-03-23T07:56:22.027442+0000","last_undegraded":"2024-03-23T07:56:22.027442+0000","last_fullsized":"2024-03-23T07:56:22.027442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096425+0000","last_change":"2024-03-23T07:56:17.740008+0000","last_active":"2024-03-23T07:56:22.096425+0000","last_peered":"2024-03-23T07:56:22.096425+0000","last_clean":"2024-03-23T07:56:22.096425+0000","last_became_active":"2024-03-23T07:56:17.739785+0000","last_became_peered":"2024-03-23T07:56:17.739785+0000","last_unstale":"2024-03-23T07:56:22.096425+0000","last_undegraded":"2024-03-23T07:56:22.096425+0000","last_fullsized":"2024-03-23T07:56:22.096425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003892+0000","last_change":"2024-03-23T07:56:17.221620+0000","last_active":"2024-03-23T07:56:22.003892+0000","last_peered":"2024-03-23T07:56:22.003892+0000","last_clean":"2024-03-23T07:56:22.003892+0000","last_became_active":"2024-03-23T07:56:17.221312+0000","last_became_peered":"2024-03-23T07:56:17.221312+0000","last_unstale":"2024-03-23T07:56:22.003892+0000","last_undegraded":"2024-03-23T07:56:22.003892+0000","last_fullsized":"2024-03-23T07:56:22.003892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:47.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.024787+0000","last_change":"2024-03-23T07:56:22.018139+0000","last_active":"2024-03-23T07:56:22.024787+0000","last_peered":"2024-03-23T07:56:22.024787+0000","last_clean":"2024-03-23T07:56:22.024787+0000","last_became_active":"2024-03-23T07:56:22.017722+0000","last_became_peered":"2024-03-23T07:56:22.017722+0000","last_unstale":"2024-03-23T07:56:22.024787+0000","last_undegraded":"2024-03-23T07:56:22.024787+0000","last_fullsized":"2024-03-23T07:56:22.024787+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983276+0000","last_change":"2024-03-23T07:56:17.729485+0000","last_active":"2024-03-23T07:56:21.983276+0000","last_peered":"2024-03-23T07:56:21.983276+0000","last_clean":"2024-03-23T07:56:21.983276+0000","last_became_active":"2024-03-23T07:56:17.729199+0000","last_became_peered":"2024-03-23T07:56:17.729199+0000","last_unstale":"2024-03-23T07:56:21.983276+0000","last_undegraded":"2024-03-23T07:56:21.983276+0000","last_fullsized":"2024-03-23T07:56:21.983276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:30:59.262864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014955+0000","last_change":"2024-03-23T07:56:17.222373+0000","last_active":"2024-03-23T07:56:22.014955+0000","last_peered":"2024-03-23T07:56:22.014955+0000","last_clean":"2024-03-23T07:56:22.014955+0000","last_became_active":"2024-03-23T07:56:17.222028+0000","last_became_peered":"2024-03-23T07:56:17.222028+0000","last_unstale":"2024-03-23T07:56:22.014955+0000","last_undegraded":"2024-03-23T07:56:22.014955+0000","last_fullsized":"2024-03-23T07:56:22.014955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.023027+0000","last_change":"2024-03-23T07:56:22.018041+0000","last_active":"2024-03-23T07:56:22.023027+0000","last_peered":"2024-03-23T07:56:22.023027+0000","last_clean":"2024-03-23T07:56:22.023027+0000","last_became_active":"2024-03-23T07:56:22.017588+0000","last_became_peered":"2024-03-23T07:56:22.017588+0000","last_unstale":"2024-03-23T07:56:22.023027+0000","last_undegraded":"2024-03-23T07:56:22.023027+0000","last_fullsized":"2024-03-23T07:56:22.023027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001772+0000","last_change":"2024-03-23T07:56:17.724698+0000","last_active":"2024-03-23T07:56:22.001772+0000","last_peered":"2024-03-23T07:56:22.001772+0000","last_clean":"2024-03-23T07:56:22.001772+0000","last_became_active":"2024-03-23T07:56:17.724382+0000","last_became_peered":"2024-03-23T07:56:17.724382+0000","last_unstale":"2024-03-23T07:56:22.001772+0000","last_undegraded":"2024-03-23T07:56:22.001772+0000","last_fullsized":"2024-03-23T07:56:22.001772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002505+0000","last_change":"2024-03-23T07:56:17.223691+0000","last_active":"2024-03-23T07:56:22.002505+0000","last_peered":"2024-03-23T07:56:22.002505+0000","last_clean":"2024-03-23T07:56:22.002505+0000","last_became_active":"2024-03-23T07:56:17.222061+0000","last_became_peered":"2024-03-23T07:56:17.222061+0000","last_unstale":"2024-03-23T07:56:22.002505+0000","last_undegraded":"2024-03-23T07:56:22.002505+0000","last_fullsized":"2024-03-23T07:56:22.002505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:00.798128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471272+0000","last_change":"2024-03-23T07:56:22.011743+0000","last_active":"2024-03-23T07:56:22.471272+0000","last_peered":"2024-03-23T07:56:22.471272+0000","last_clean":"2024-03-23T07:56:22.471272+0000","last_became_active":"2024-03-23T07:56:22.011280+0000","last_became_peered":"2024-03-23T07:56:22.011280+0000","last_unstale":"2024-03-23T07:56:22.471272+0000","last_undegraded":"2024-03-23T07:56:22.471272+0000","last_fullsized":"2024-03-23T07:56:22.471272+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:05.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988647+0000","last_change":"2024-03-23T07:56:17.722136+0000","last_active":"2024-03-23T07:56:21.988647+0000","last_peered":"2024-03-23T07:56:21.988647+0000","last_clean":"2024-03-23T07:56:21.988647+0000","last_became_active":"2024-03-23T07:56:17.720479+0000","last_became_peered":"2024-03-23T07:56:17.720479+0000","last_unstale":"2024-03-23T07:56:21.988647+0000","last_undegraded":"2024-03-23T07:56:21.988647+0000","last_fullsized":"2024-03-23T07:56:21.988647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003825+0000","last_change":"2024-03-23T07:56:17.227266+0000","last_active":"2024-03-23T07:56:22.003825+0000","last_peered":"2024-03-23T07:56:22.003825+0000","last_clean":"2024-03-23T07:56:22.003825+0000","last_became_active":"2024-03-23T07:56:17.226775+0000","last_became_peered":"2024-03-23T07:56:17.226775+0000","last_unstale":"2024-03-23T07:56:22.003825+0000","last_undegraded":"2024-03-23T07:56:22.003825+0000","last_fullsized":"2024-03-23T07:56:22.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005759+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:56:22.005759+0000","last_peered":"2024-03-23T07:56:22.005759+0000","last_clean":"2024-03-23T07:56:22.005759+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:56:22.005759+0000","last_undegraded":"2024-03-23T07:56:22.005759+0000","last_fullsized":"2024-03-23T07:56:22.005759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097280+0000","last_change":"2024-03-23T07:56:22.024202+0000","last_active":"2024-03-23T07:56:22.097280+0000","last_peered":"2024-03-23T07:56:22.097280+0000","last_clean":"2024-03-23T07:56:22.097280+0000","last_became_active":"2024-03-23T07:56:22.018941+0000","last_became_peered":"2024-03-23T07:56:22.018941+0000","last_unstale":"2024-03-23T07:56:22.097280+0000","last_undegraded":"2024-03-23T07:56:22.097280+0000","last_fullsized":"2024-03-23T07:56:22.097280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:25.917706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097267+0000","last_change":"2024-03-23T07:56:17.734848+0000","last_active":"2024-03-23T07:56:22.097267+0000","last_peered":"2024-03-23T07:56:22.097267+0000","last_clean":"2024-03-23T07:56:22.097267+0000","last_became_active":"2024-03-23T07:56:17.733233+0000","last_became_peered":"2024-03-23T07:56:17.733233+0000","last_unstale":"2024-03-23T07:56:22.097267+0000","last_undegraded":"2024-03-23T07:56:22.097267+0000","last_fullsized":"2024-03-23T07:56:22.097267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003338+0000","last_change":"2024-03-23T07:56:17.215068+0000","last_active":"2024-03-23T07:56:22.003338+0000","last_peered":"2024-03-23T07:56:22.003338+0000","last_clean":"2024-03-23T07:56:22.003338+0000","last_became_active":"2024-03-23T07:56:17.214726+0000","last_became_peered":"2024-03-23T07:56:17.214726+0000","last_unstale":"2024-03-23T07:56:22.003338+0000","last_undegraded":"2024-03-23T07:56:22.003338+0000","last_fullsized":"2024-03-23T07:56:22.003338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093494+0000","last_change":"2024-03-23T07:56:22.023739+0000","last_active":"2024-03-23T07:56:22.093494+0000","last_peered":"2024-03-23T07:56:22.093494+0000","last_clean":"2024-03-23T07:56:22.093494+0000","last_became_active":"2024-03-23T07:56:22.023366+0000","last_became_peered":"2024-03-23T07:56:22.023366+0000","last_unstale":"2024-03-23T07:56:22.093494+0000","last_undegraded":"2024-03-23T07:56:22.093494+0000","last_fullsized":"2024-03-23T07:56:22.093494+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:13.143472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096511+0000","last_change":"2024-03-23T07:56:17.737947+0000","last_active":"2024-03-23T07:56:22.096511+0000","last_peered":"2024-03-23T07:56:22.096511+0000","last_clean":"2024-03-23T07:56:22.096511+0000","last_became_active":"2024-03-23T07:56:17.733346+0000","last_became_peered":"2024-03-23T07:56:17.733346+0000","last_unstale":"2024-03-23T07:56:22.096511+0000","last_undegraded":"2024-03-23T07:56:22.096511+0000","last_fullsized":"2024-03-23T07:56:22.096511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998748+0000","last_change":"2024-03-23T07:56:17.221283+0000","last_active":"2024-03-23T07:56:21.998748+0000","last_peered":"2024-03-23T07:56:21.998748+0000","last_clean":"2024-03-23T07:56:21.998748+0000","last_became_active":"2024-03-23T07:56:17.219961+0000","last_became_peered":"2024-03-23T07:56:17.219961+0000","last_unstale":"2024-03-23T07:56:21.998748+0000","last_undegraded":"2024-03-23T07:56:21.998748+0000","last_fullsized":"2024-03-23T07:56:21.998748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:04:11.860279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.470374+0000","last_change":"2024-03-23T07:56:22.016551+0000","last_active":"2024-03-23T07:56:22.470374+0000","last_peered":"2024-03-23T07:56:22.470374+0000","last_clean":"2024-03-23T07:56:22.470374+0000","last_became_active":"2024-03-23T07:56:22.016136+0000","last_became_peered":"2024-03-23T07:56:22.016136+0000","last_unstale":"2024-03-23T07:56:22.470374+0000","last_undegraded":"2024-03-23T07:56:22.470374+0000","last_fullsized":"2024-03-23T07:56:22.470374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:36.873288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096655+0000","last_change":"2024-03-23T07:56:17.743611+0000","last_active":"2024-03-23T07:56:22.096655+0000","last_peered":"2024-03-23T07:56:22.096655+0000","last_clean":"2024-03-23T07:56:22.096655+0000","last_became_active":"2024-03-23T07:56:17.742926+0000","last_became_peered":"2024-03-23T07:56:17.742926+0000","last_unstale":"2024-03-23T07:56:22.096655+0000","last_undegraded":"2024-03-23T07:56:22.096655+0000","last_fullsized":"2024-03-23T07:56:22.096655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994328+0000","last_change":"2024-03-23T07:56:17.219709+0000","last_active":"2024-03-23T07:56:21.994328+0000","last_peered":"2024-03-23T07:56:21.994328+0000","last_clean":"2024-03-23T07:56:21.994328+0000","last_became_active":"2024-03-23T07:56:17.218465+0000","last_became_peered":"2024-03-23T07:56:17.218465+0000","last_unstale":"2024-03-23T07:56:21.994328+0000","last_undegraded":"2024-03-23T07:56:21.994328+0000","last_fullsized":"2024-03-23T07:56:21.994328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096964+0000","last_change":"2024-03-23T07:56:22.017346+0000","last_active":"2024-03-23T07:56:22.096964+0000","last_peered":"2024-03-23T07:56:22.096964+0000","last_clean":"2024-03-23T07:56:22.096964+0000","last_became_active":"2024-03-23T07:56:22.016949+0000","last_became_peered":"2024-03-23T07:56:22.016949+0000","last_unstale":"2024-03-23T07:56:22.096964+0000","last_undegraded":"2024-03-23T07:56:22.096964+0000","last_fullsized":"2024-03-23T07:56:22.096964+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:45.299178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093733+0000","last_change":"2024-03-23T07:56:17.742996+0000","last_active":"2024-03-23T07:56:22.093733+0000","last_peered":"2024-03-23T07:56:22.093733+0000","last_clean":"2024-03-23T07:56:22.093733+0000","last_became_active":"2024-03-23T07:56:17.742660+0000","last_became_peered":"2024-03-23T07:56:17.742660+0000","last_unstale":"2024-03-23T07:56:22.093733+0000","last_undegraded":"2024-03-23T07:56:22.093733+0000","last_fullsized":"2024-03-23T07:56:22.093733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004353+0000","last_change":"2024-03-23T07:56:17.215901+0000","last_active":"2024-03-23T07:56:22.004353+0000","last_peered":"2024-03-23T07:56:22.004353+0000","last_clean":"2024-03-23T07:56:22.004353+0000","last_became_active":"2024-03-23T07:56:17.215638+0000","last_became_peered":"2024-03-23T07:56:17.215638+0000","last_unstale":"2024-03-23T07:56:22.004353+0000","last_undegraded":"2024-03-23T07:56:22.004353+0000","last_fullsized":"2024-03-23T07:56:22.004353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093529+0000","last_change":"2024-03-23T07:56:22.010512+0000","last_active":"2024-03-23T07:56:22.093529+0000","last_peered":"2024-03-23T07:56:22.093529+0000","last_clean":"2024-03-23T07:56:22.093529+0000","last_became_active":"2024-03-23T07:56:22.009577+0000","last_became_peered":"2024-03-23T07:56:22.009577+0000","last_unstale":"2024-03-23T07:56:22.093529+0000","last_undegraded":"2024-03-23T07:56:22.093529+0000","last_fullsized":"2024-03-23T07:56:22.093529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:24.844391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097072+0000","last_change":"2024-03-23T07:56:17.732277+0000","last_active":"2024-03-23T07:56:22.097072+0000","last_peered":"2024-03-23T07:56:22.097072+0000","last_clean":"2024-03-23T07:56:22.097072+0000","last_became_active":"2024-03-23T07:56:17.731487+0000","last_became_peered":"2024-03-23T07:56:17.731487+0000","last_unstale":"2024-03-23T07:56:22.097072+0000","last_undegraded":"2024-03-23T07:56:22.097072+0000","last_fullsized":"2024-03-23T07:56:22.097072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:39.462067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985538+0000","last_change":"2024-03-23T07:56:17.211010+0000","last_active":"2024-03-23T07:56:21.985538+0000","last_peered":"2024-03-23T07:56:21.985538+0000","last_clean":"2024-03-23T07:56:21.985538+0000","last_became_active":"2024-03-23T07:56:17.210710+0000","last_became_peered":"2024-03-23T07:56:17.210710+0000","last_unstale":"2024-03-23T07:56:21.985538+0000","last_undegraded":"2024-03-23T07:56:21.985538+0000","last_fullsized":"2024-03-23T07:56:21.985538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471596+0000","last_change":"2024-03-23T07:56:22.019325+0000","last_active":"2024-03-23T07:56:22.471596+0000","last_peered":"2024-03-23T07:56:22.471596+0000","last_clean":"2024-03-23T07:56:22.471596+0000","last_became_active":"2024-03-23T07:56:22.016793+0000","last_became_peered":"2024-03-23T07:56:22.016793+0000","last_unstale":"2024-03-23T07:56:22.471596+0000","last_undegraded":"2024-03-23T07:56:22.471596+0000","last_fullsized":"2024-03-23T07:56:22.471596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:08:59.171123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093481+0000","last_change":"2024-03-23T07:56:17.716144+0000","last_active":"2024-03-23T07:56:22.093481+0000","last_peered":"2024-03-23T07:56:22.093481+0000","last_clean":"2024-03-23T07:56:22.093481+0000","last_became_active":"2024-03-23T07:56:17.715899+0000","last_became_peered":"2024-03-23T07:56:17.715899+0000","last_unstale":"2024-03-23T07:56:22.093481+0000","last_undegraded":"2024-03-23T07:56:22.093481+0000","last_fullsized":"2024-03-23T07:56:22.093481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005380+0000","last_change":"2024-03-23T07:56:17.226828+0000","last_active":"2024-03-23T07:56:22.005380+0000","last_peered":"2024-03-23T07:56:22.005380+0000","last_clean":"2024-03-23T07:56:22.005380+0000","last_became_active":"2024-03-23T07:56:17.226551+0000","last_became_peered":"2024-03-23T07:56:17.226551+0000","last_unstale":"2024-03-23T07:56:22.005380+0000","last_undegraded":"2024-03-23T07:56:22.005380+0000","last_fullsized":"2024-03-23T07:56:22.005380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:03.843216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471755+0000","last_change":"2024-03-23T07:56:22.019606+0000","last_active":"2024-03-23T07:56:22.471755+0000","last_peered":"2024-03-23T07:56:22.471755+0000","last_clean":"2024-03-23T07:56:22.471755+0000","last_became_active":"2024-03-23T07:56:22.019280+0000","last_became_peered":"2024-03-23T07:56:22.019280+0000","last_unstale":"2024-03-23T07:56:22.471755+0000","last_undegraded":"2024-03-23T07:56:22.471755+0000","last_fullsized":"2024-03-23T07:56:22.471755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:28.706338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097162+0000","last_change":"2024-03-23T07:56:17.746537+0000","last_active":"2024-03-23T07:56:22.097162+0000","last_peered":"2024-03-23T07:56:22.097162+0000","last_clean":"2024-03-23T07:56:22.097162+0000","last_became_active":"2024-03-23T07:56:17.746054+0000","last_became_peered":"2024-03-23T07:56:17.746054+0000","last_unstale":"2024-03-23T07:56:22.097162+0000","last_undegraded":"2024-03-23T07:56:22.097162+0000","last_fullsized":"2024-03-23T07:56:22.097162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:09.358882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003661+0000","last_change":"2024-03-23T07:56:17.225527+0000","last_active":"2024-03-23T07:56:22.003661+0000","last_peered":"2024-03-23T07:56:22.003661+0000","last_clean":"2024-03-23T07:56:22.003661+0000","last_became_active":"2024-03-23T07:56:17.224860+0000","last_became_peered":"2024-03-23T07:56:17.224860+0000","last_unstale":"2024-03-23T07:56:22.003661+0000","last_undegraded":"2024-03-23T07:56:22.003661+0000","last_fullsized":"2024-03-23T07:56:22.003661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:26.692314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028094+0000","last_change":"2024-03-23T07:56:22.023659+0000","last_active":"2024-03-23T07:56:22.028094+0000","last_peered":"2024-03-23T07:56:22.028094+0000","last_clean":"2024-03-23T07:56:22.028094+0000","last_became_active":"2024-03-23T07:56:22.023230+0000","last_became_peered":"2024-03-23T07:56:22.023230+0000","last_unstale":"2024-03-23T07:56:22.028094+0000","last_undegraded":"2024-03-23T07:56:22.028094+0000","last_fullsized":"2024-03-23T07:56:22.028094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002303+0000","last_change":"2024-03-23T07:56:17.729587+0000","last_active":"2024-03-23T07:56:22.002303+0000","last_peered":"2024-03-23T07:56:22.002303+0000","last_clean":"2024-03-23T07:56:22.002303+0000","last_became_active":"2024-03-23T07:56:17.727873+0000","last_became_peered":"2024-03-23T07:56:17.727873+0000","last_unstale":"2024-03-23T07:56:22.002303+0000","last_undegraded":"2024-03-23T07:56:22.002303+0000","last_fullsized":"2024-03-23T07:56:22.002303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017227+0000","last_change":"2024-03-23T07:56:17.215408+0000","last_active":"2024-03-23T07:56:22.017227+0000","last_peered":"2024-03-23T07:56:22.017227+0000","last_clean":"2024-03-23T07:56:22.017227+0000","last_became_active":"2024-03-23T07:56:17.215155+0000","last_became_peered":"2024-03-23T07:56:17.215155+0000","last_unstale":"2024-03-23T07:56:22.017227+0000","last_undegraded":"2024-03-23T07:56:22.017227+0000","last_fullsized":"2024-03-23T07:56:22.017227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097238+0000","last_change":"2024-03-23T07:56:22.010493+0000","last_active":"2024-03-23T07:56:22.097238+0000","last_peered":"2024-03-23T07:56:22.097238+0000","last_clean":"2024-03-23T07:56:22.097238+0000","last_became_active":"2024-03-23T07:56:22.010048+0000","last_became_peered":"2024-03-23T07:56:22.010048+0000","last_unstale":"2024-03-23T07:56:22.097238+0000","last_undegraded":"2024-03-23T07:56:22.097238+0000","last_fullsized":"2024-03-23T07:56:22.097238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:17.490111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096554+0000","last_change":"2024-03-23T07:56:17.735587+0000","last_active":"2024-03-23T07:56:22.096554+0000","last_peered":"2024-03-23T07:56:22.096554+0000","last_clean":"2024-03-23T07:56:22.096554+0000","last_became_active":"2024-03-23T07:56:17.735319+0000","last_became_peered":"2024-03-23T07:56:17.735319+0000","last_unstale":"2024-03-23T07:56:22.096554+0000","last_undegraded":"2024-03-23T07:56:22.096554+0000","last_fullsized":"2024-03-23T07:56:22.096554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:36.310952+0000","last_change":"2024-03-23T07:56:17.217671+0000","last_active":"2024-03-23T07:56:36.310952+0000","last_peered":"2024-03-23T07:56:36.310952+0000","last_clean":"2024-03-23T07:56:36.310952+0000","last_became_active":"2024-03-23T07:56:17.217332+0000","last_became_peered":"2024-03-23T07:56:17.217332+0000","last_unstale":"2024-03-23T07:56:36.310952+0000","last_undegraded":"2024-03-23T07:56:36.310952+0000","last_fullsized":"2024-03-23T07:56:36.310952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:40:30.911275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018197+0000","last_change":"2024-03-23T07:56:22.013208+0000","last_active":"2024-03-23T07:56:22.018197+0000","last_peered":"2024-03-23T07:56:22.018197+0000","last_clean":"2024-03-23T07:56:22.018197+0000","last_became_active":"2024-03-23T07:56:22.012577+0000","last_became_peered":"2024-03-23T07:56:22.012577+0000","last_unstale":"2024-03-23T07:56:22.018197+0000","last_undegraded":"2024-03-23T07:56:22.018197+0000","last_fullsized":"2024-03-23T07:56:22.018197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097206+0000","last_change":"2024-03-23T07:56:17.735251+0000","last_active":"2024-03-23T07:56:22.097206+0000","last_peered":"2024-03-23T07:56:22.097206+0000","last_clean":"2024-03-23T07:56:22.097206+0000","last_became_active":"2024-03-23T07:56:17.731173+0000","last_became_peered":"2024-03-23T07:56:17.731173+0000","last_unstale":"2024-03-23T07:56:22.097206+0000","last_undegraded":"2024-03-23T07:56:22.097206+0000","last_fullsized":"2024-03-23T07:56:22.097206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:54.379935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018228+0000","last_change":"2024-03-23T07:56:17.216512+0000","last_active":"2024-03-23T07:56:22.018228+0000","last_peered":"2024-03-23T07:56:22.018228+0000","last_clean":"2024-03-23T07:56:22.018228+0000","last_became_active":"2024-03-23T07:56:17.215743+0000","last_became_peered":"2024-03-23T07:56:17.215743+0000","last_unstale":"2024-03-23T07:56:22.018228+0000","last_undegraded":"2024-03-23T07:56:22.018228+0000","last_fullsized":"2024-03-23T07:56:22.018228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093611+0000","last_change":"2024-03-23T07:56:22.014702+0000","last_active":"2024-03-23T07:56:22.093611+0000","last_peered":"2024-03-23T07:56:22.093611+0000","last_clean":"2024-03-23T07:56:22.093611+0000","last_became_active":"2024-03-23T07:56:22.014290+0000","last_became_peered":"2024-03-23T07:56:22.014290+0000","last_unstale":"2024-03-23T07:56:22.093611+0000","last_undegraded":"2024-03-23T07:56:22.093611+0000","last_fullsized":"2024-03-23T07:56:22.093611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005107+0000","last_change":"2024-03-23T07:56:17.725552+0000","last_active":"2024-03-23T07:56:22.005107+0000","last_peered":"2024-03-23T07:56:22.005107+0000","last_clean":"2024-03-23T07:56:22.005107+0000","last_became_active":"2024-03-23T07:56:17.725241+0000","last_became_peered":"2024-03-23T07:56:17.725241+0000","last_unstale":"2024-03-23T07:56:22.005107+0000","last_undegraded":"2024-03-23T07:56:22.005107+0000","last_fullsized":"2024-03-23T07:56:22.005107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:08:14.226813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.999373+0000","last_change":"2024-03-23T07:56:17.224220+0000","last_active":"2024-03-23T07:56:21.999373+0000","last_peered":"2024-03-23T07:56:21.999373+0000","last_clean":"2024-03-23T07:56:21.999373+0000","last_became_active":"2024-03-23T07:56:17.223476+0000","last_became_peered":"2024-03-23T07:56:17.223476+0000","last_unstale":"2024-03-23T07:56:21.999373+0000","last_undegraded":"2024-03-23T07:56:21.999373+0000","last_fullsized":"2024-03-23T07:56:21.999373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031049+0000","last_change":"2024-03-23T07:56:22.027325+0000","last_active":"2024-03-23T07:56:22.031049+0000","last_peered":"2024-03-23T07:56:22.031049+0000","last_clean":"2024-03-23T07:56:22.031049+0000","last_became_active":"2024-03-23T07:56:22.026858+0000","last_became_peered":"2024-03-23T07:56:22.026858+0000","last_unstale":"2024-03-23T07:56:22.031049+0000","last_undegraded":"2024-03-23T07:56:22.031049+0000","last_fullsized":"2024-03-23T07:56:22.031049+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:38.880449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093719+0000","last_change":"2024-03-23T07:56:17.736540+0000","last_active":"2024-03-23T07:56:22.093719+0000","last_peered":"2024-03-23T07:56:22.093719+0000","last_clean":"2024-03-23T07:56:22.093719+0000","last_became_active":"2024-03-23T07:56:17.727131+0000","last_became_peered":"2024-03-23T07:56:17.727131+0000","last_unstale":"2024-03-23T07:56:22.093719+0000","last_undegraded":"2024-03-23T07:56:22.093719+0000","last_fullsized":"2024-03-23T07:56:22.093719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.006955+0000","last_change":"2024-03-23T07:56:17.215558+0000","last_active":"2024-03-23T07:56:22.006955+0000","last_peered":"2024-03-23T07:56:22.006955+0000","last_clean":"2024-03-23T07:56:22.006955+0000","last_became_active":"2024-03-23T07:56:17.214342+0000","last_became_peered":"2024-03-23T07:56:17.214342+0000","last_unstale":"2024-03-23T07:56:22.006955+0000","last_undegraded":"2024-03-23T07:56:22.006955+0000","last_fullsized":"2024-03-23T07:56:22.006955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.010553+0000","last_change":"2024-03-23T07:56:22.008000+0000","last_active":"2024-03-23T07:56:22.010553+0000","last_peered":"2024-03-23T07:56:22.010553+0000","last_clean":"2024-03-23T07:56:22.010553+0000","last_became_active":"2024-03-23T07:56:22.006996+0000","last_became_peered":"2024-03-23T07:56:22.006996+0000","last_unstale":"2024-03-23T07:56:22.010553+0000","last_undegraded":"2024-03-23T07:56:22.010553+0000","last_fullsized":"2024-03-23T07:56:22.010553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:24:55.446700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015561+0000","last_change":"2024-03-23T07:56:22.009204+0000","last_active":"2024-03-23T07:56:22.015561+0000","last_peered":"2024-03-23T07:56:22.015561+0000","last_clean":"2024-03-23T07:56:22.015561+0000","last_became_active":"2024-03-23T07:56:22.008744+0000","last_became_peered":"2024-03-23T07:56:22.008744+0000","last_unstale":"2024-03-23T07:56:22.015561+0000","last_undegraded":"2024-03-23T07:56:22.015561+0000","last_fullsized":"2024-03-23T07:56:22.015561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:28.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984217+0000","last_change":"2024-03-23T07:56:17.733961+0000","last_active":"2024-03-23T07:56:21.984217+0000","last_peered":"2024-03-23T07:56:21.984217+0000","last_clean":"2024-03-23T07:56:21.984217+0000","last_became_active":"2024-03-23T07:56:17.733646+0000","last_became_peered":"2024-03-23T07:56:17.733646+0000","last_unstale":"2024-03-23T07:56:21.984217+0000","last_undegraded":"2024-03-23T07:56:21.984217+0000","last_fullsized":"2024-03-23T07:56:21.984217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002413+0000","last_change":"2024-03-23T07:56:17.223808+0000","last_active":"2024-03-23T07:56:22.002413+0000","last_peered":"2024-03-23T07:56:22.002413+0000","last_clean":"2024-03-23T07:56:22.002413+0000","last_became_active":"2024-03-23T07:56:17.223429+0000","last_became_peered":"2024-03-23T07:56:17.223429+0000","last_unstale":"2024-03-23T07:56:22.002413+0000","last_undegraded":"2024-03-23T07:56:22.002413+0000","last_fullsized":"2024-03-23T07:56:22.002413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:57.800903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097333+0000","last_change":"2024-03-23T07:56:22.015838+0000","last_active":"2024-03-23T07:56:22.097333+0000","last_peered":"2024-03-23T07:56:22.097333+0000","last_clean":"2024-03-23T07:56:22.097333+0000","last_became_active":"2024-03-23T07:56:22.015393+0000","last_became_peered":"2024-03-23T07:56:22.015393+0000","last_unstale":"2024-03-23T07:56:22.097333+0000","last_undegraded":"2024-03-23T07:56:22.097333+0000","last_fullsized":"2024-03-23T07:56:22.097333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:35:13.840057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985820+0000","last_change":"2024-03-23T07:56:17.729460+0000","last_active":"2024-03-23T07:56:21.985820+0000","last_peered":"2024-03-23T07:56:21.985820+0000","last_clean":"2024-03-23T07:56:21.985820+0000","last_became_active":"2024-03-23T07:56:17.728590+0000","last_became_peered":"2024-03-23T07:56:17.728590+0000","last_unstale":"2024-03-23T07:56:21.985820+0000","last_undegraded":"2024-03-23T07:56:21.985820+0000","last_fullsized":"2024-03-23T07:56:21.985820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:32:18.762479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001880+0000","last_change":"2024-03-23T07:56:17.216006+0000","last_active":"2024-03-23T07:56:22.001880+0000","last_peered":"2024-03-23T07:56:22.001880+0000","last_clean":"2024-03-23T07:56:22.001880+0000","last_became_active":"2024-03-23T07:56:17.215734+0000","last_became_peered":"2024-03-23T07:56:17.215734+0000","last_unstale":"2024-03-23T07:56:22.001880+0000","last_undegraded":"2024-03-23T07:56:22.001880+0000","last_fullsized":"2024-03-23T07:56:22.001880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003800+0000","last_change":"2024-03-23T07:56:17.225732+0000","last_active":"2024-03-23T07:56:22.003800+0000","last_peered":"2024-03-23T07:56:22.003800+0000","last_clean":"2024-03-23T07:56:22.003800+0000","last_became_active":"2024-03-23T07:56:17.225216+0000","last_became_peered":"2024-03-23T07:56:17.225216+0000","last_unstale":"2024-03-23T07:56:22.003800+0000","last_undegraded":"2024-03-23T07:56:22.003800+0000","last_fullsized":"2024-03-23T07:56:22.003800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:08.861374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982688+0000","last_change":"2024-03-23T07:56:17.750834+0000","last_active":"2024-03-23T07:56:21.982688+0000","last_peered":"2024-03-23T07:56:21.982688+0000","last_clean":"2024-03-23T07:56:21.982688+0000","last_became_active":"2024-03-23T07:56:17.750483+0000","last_became_peered":"2024-03-23T07:56:17.750483+0000","last_unstale":"2024-03-23T07:56:21.982688+0000","last_undegraded":"2024-03-23T07:56:21.982688+0000","last_fullsized":"2024-03-23T07:56:21.982688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018371+0000","last_change":"2024-03-23T07:56:22.013558+0000","last_active":"2024-03-23T07:56:22.018371+0000","last_peered":"2024-03-23T07:56:22.018371+0000","last_clean":"2024-03-23T07:56:22.018371+0000","last_became_active":"2024-03-23T07:56:22.011671+0000","last_became_peered":"2024-03-23T07:56:22.011671+0000","last_unstale":"2024-03-23T07:56:22.018371+0000","last_undegraded":"2024-03-23T07:56:22.018371+0000","last_fullsized":"2024-03-23T07:56:22.018371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:07.197610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991122+0000","last_change":"2024-03-23T07:56:17.217610+0000","last_active":"2024-03-23T07:56:21.991122+0000","last_peered":"2024-03-23T07:56:21.991122+0000","last_clean":"2024-03-23T07:56:21.991122+0000","last_became_active":"2024-03-23T07:56:17.217342+0000","last_became_peered":"2024-03-23T07:56:17.217342+0000","last_unstale":"2024-03-23T07:56:21.991122+0000","last_undegraded":"2024-03-23T07:56:21.991122+0000","last_fullsized":"2024-03-23T07:56:21.991122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:27.578107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093588+0000","last_change":"2024-03-23T07:56:17.719621+0000","last_active":"2024-03-23T07:56:22.093588+0000","last_peered":"2024-03-23T07:56:22.093588+0000","last_clean":"2024-03-23T07:56:22.093588+0000","last_became_active":"2024-03-23T07:56:17.719296+0000","last_became_peered":"2024-03-23T07:56:17.719296+0000","last_unstale":"2024-03-23T07:56:22.093588+0000","last_undegraded":"2024-03-23T07:56:22.093588+0000","last_fullsized":"2024-03-23T07:56:22.093588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031093+0000","last_change":"2024-03-23T07:56:22.027765+0000","last_active":"2024-03-23T07:56:22.031093+0000","last_peered":"2024-03-23T07:56:22.031093+0000","last_clean":"2024-03-23T07:56:22.031093+0000","last_became_active":"2024-03-23T07:56:22.027420+0000","last_became_peered":"2024-03-23T07:56:22.027420+0000","last_unstale":"2024-03-23T07:56:22.031093+0000","last_undegraded":"2024-03-23T07:56:22.031093+0000","last_fullsized":"2024-03-23T07:56:22.031093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:25.578708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093412+0000","last_change":"2024-03-23T07:56:17.721639+0000","last_active":"2024-03-23T07:56:22.093412+0000","last_peered":"2024-03-23T07:56:22.093412+0000","last_clean":"2024-03-23T07:56:22.093412+0000","last_became_active":"2024-03-23T07:56:17.721372+0000","last_became_peered":"2024-03-23T07:56:17.721372+0000","last_unstale":"2024-03-23T07:56:22.093412+0000","last_undegraded":"2024-03-23T07:56:22.093412+0000","last_fullsized":"2024-03-23T07:56:22.093412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014858+0000","last_change":"2024-03-23T07:56:17.741387+0000","last_active":"2024-03-23T07:56:22.014858+0000","last_peered":"2024-03-23T07:56:22.014858+0000","last_clean":"2024-03-23T07:56:22.014858+0000","last_became_active":"2024-03-23T07:56:17.741091+0000","last_became_peered":"2024-03-23T07:56:17.741091+0000","last_unstale":"2024-03-23T07:56:22.014858+0000","last_undegraded":"2024-03-23T07:56:22.014858+0000","last_fullsized":"2024-03-23T07:56:22.014858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097117+0000","last_change":"2024-03-23T07:56:17.729701+0000","last_active":"2024-03-23T07:56:22.097117+0000","last_peered":"2024-03-23T07:56:22.097117+0000","last_clean":"2024-03-23T07:56:22.097117+0000","last_became_active":"2024-03-23T07:56:17.729481+0000","last_became_peered":"2024-03-23T07:56:17.729481+0000","last_unstale":"2024-03-23T07:56:22.097117+0000","last_undegraded":"2024-03-23T07:56:22.097117+0000","last_fullsized":"2024-03-23T07:56:22.097117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.990887+0000","last_change":"2024-03-23T07:56:17.724914+0000","last_active":"2024-03-23T07:56:21.990887+0000","last_peered":"2024-03-23T07:56:21.990887+0000","last_clean":"2024-03-23T07:56:21.990887+0000","last_became_active":"2024-03-23T07:56:17.724627+0000","last_became_peered":"2024-03-23T07:56:17.724627+0000","last_unstale":"2024-03-23T07:56:21.990887+0000","last_undegraded":"2024-03-23T07:56:21.990887+0000","last_fullsized":"2024-03-23T07:56:21.990887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:36.852829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002999+0000","last_change":"2024-03-23T07:56:17.745053+0000","last_active":"2024-03-23T07:56:22.002999+0000","last_peered":"2024-03-23T07:56:22.002999+0000","last_clean":"2024-03-23T07:56:22.002999+0000","last_became_active":"2024-03-23T07:56:17.744775+0000","last_became_peered":"2024-03-23T07:56:17.744775+0000","last_unstale":"2024-03-23T07:56:22.002999+0000","last_undegraded":"2024-03-23T07:56:22.002999+0000","last_fullsized":"2024-03-23T07:56:22.002999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001976+0000","last_change":"2024-03-23T07:56:17.726849+0000","last_active":"2024-03-23T07:56:22.001976+0000","last_peered":"2024-03-23T07:56:22.001976+0000","last_clean":"2024-03-23T07:56:22.001976+0000","last_became_active":"2024-03-23T07:56:17.726339+0000","last_became_peered":"2024-03-23T07:56:17.726339+0000","last_unstale":"2024-03-23T07:56:22.001976+0000","last_undegraded":"2024-03-23T07:56:22.001976+0000","last_fullsized":"2024-03-23T07:56:22.001976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:19:42.241859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093701+0000","last_change":"2024-03-23T07:56:17.736674+0000","last_active":"2024-03-23T07:56:22.093701+0000","last_peered":"2024-03-23T07:56:22.093701+0000","last_clean":"2024-03-23T07:56:22.093701+0000","last_became_active":"2024-03-23T07:56:17.728841+0000","last_became_peered":"2024-03-23T07:56:17.728841+0000","last_unstale":"2024-03-23T07:56:22.093701+0000","last_undegraded":"2024-03-23T07:56:22.093701+0000","last_fullsized":"2024-03-23T07:56:22.093701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000680+0000","last_change":"2024-03-23T07:56:17.734249+0000","last_active":"2024-03-23T07:56:22.000680+0000","last_peered":"2024-03-23T07:56:22.000680+0000","last_clean":"2024-03-23T07:56:22.000680+0000","last_became_active":"2024-03-23T07:56:17.733805+0000","last_became_peered":"2024-03-23T07:56:17.733805+0000","last_unstale":"2024-03-23T07:56:22.000680+0000","last_undegraded":"2024-03-23T07:56:22.000680+0000","last_fullsized":"2024-03-23T07:56:22.000680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:47.287496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097031+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097031+0000","last_peered":"2024-03-23T07:56:22.097031+0000","last_clean":"2024-03-23T07:56:22.097031+0000","last_became_active":"2024-03-23T07:56:17.727006+0000","last_became_peered":"2024-03-23T07:56:17.727006+0000","last_unstale":"2024-03-23T07:56:22.097031+0000","last_undegraded":"2024-03-23T07:56:22.097031+0000","last_fullsized":"2024-03-23T07:56:22.097031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:54.056625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.989132+0000","last_change":"2024-03-23T07:56:17.730066+0000","last_active":"2024-03-23T07:56:21.989132+0000","last_peered":"2024-03-23T07:56:21.989132+0000","last_clean":"2024-03-23T07:56:21.989132+0000","last_became_active":"2024-03-23T07:56:17.729250+0000","last_became_peered":"2024-03-23T07:56:17.729250+0000","last_unstale":"2024-03-23T07:56:21.989132+0000","last_undegraded":"2024-03-23T07:56:21.989132+0000","last_fullsized":"2024-03-23T07:56:21.989132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:28.507099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093550+0000","last_change":"2024-03-23T07:56:17.732206+0000","last_active":"2024-03-23T07:56:22.093550+0000","last_peered":"2024-03-23T07:56:22.093550+0000","last_clean":"2024-03-23T07:56:22.093550+0000","last_became_active":"2024-03-23T07:56:17.731391+0000","last_became_peered":"2024-03-23T07:56:17.731391+0000","last_unstale":"2024-03-23T07:56:22.093550+0000","last_undegraded":"2024-03-23T07:56:22.093550+0000","last_fullsized":"2024-03-23T07:56:22.093550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982447+0000","last_change":"2024-03-23T07:56:17.727582+0000","last_active":"2024-03-23T07:56:21.982447+0000","last_peered":"2024-03-23T07:56:21.982447+0000","last_clean":"2024-03-23T07:56:21.982447+0000","last_became_active":"2024-03-23T07:56:17.727312+0000","last_became_peered":"2024-03-23T07:56:17.727312+0000","last_unstale":"2024-03-23T07:56:21.982447+0000","last_undegraded":"2024-03-23T07:56:21.982447+0000","last_fullsized":"2024-03-23T07:56:21.982447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096494+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096494+0000","last_peered":"2024-03-23T07:56:22.096494+0000","last_clean":"2024-03-23T07:56:22.096494+0000","last_became_active":"2024-03-23T07:56:17.743717+0000","last_became_peered":"2024-03-23T07:56:17.743717+0000","last_unstale":"2024-03-23T07:56:22.096494+0000","last_undegraded":"2024-03-23T07:56:22.096494+0000","last_fullsized":"2024-03-23T07:56:22.096494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:35.095859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097297+0000","last_change":"2024-03-23T07:56:17.729192+0000","last_active":"2024-03-23T07:56:22.097297+0000","last_peered":"2024-03-23T07:56:22.097297+0000","last_clean":"2024-03-23T07:56:22.097297+0000","last_became_active":"2024-03-23T07:56:17.728922+0000","last_became_peered":"2024-03-23T07:56:17.728922+0000","last_unstale":"2024-03-23T07:56:22.097297+0000","last_undegraded":"2024-03-23T07:56:22.097297+0000","last_fullsized":"2024-03-23T07:56:22.097297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017348+0000","last_change":"2024-03-23T07:56:17.731099+0000","last_active":"2024-03-23T07:56:22.017348+0000","last_peered":"2024-03-23T07:56:22.017348+0000","last_clean":"2024-03-23T07:56:22.017348+0000","last_became_active":"2024-03-23T07:56:17.730597+0000","last_became_peered":"2024-03-23T07:56:17.730597+0000","last_unstale":"2024-03-23T07:56:22.017348+0000","last_undegraded":"2024-03-23T07:56:22.017348+0000","last_fullsized":"2024-03-23T07:56:22.017348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097349+0000","last_change":"2024-03-23T07:56:17.748094+0000","last_active":"2024-03-23T07:56:22.097349+0000","last_peered":"2024-03-23T07:56:22.097349+0000","last_clean":"2024-03-23T07:56:22.097349+0000","last_became_active":"2024-03-23T07:56:17.747867+0000","last_became_peered":"2024-03-23T07:56:17.747867+0000","last_unstale":"2024-03-23T07:56:22.097349+0000","last_undegraded":"2024-03-23T07:56:22.097349+0000","last_fullsized":"2024-03-23T07:56:22.097349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:53.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097004+0000","last_change":"2024-03-23T07:56:17.731875+0000","last_active":"2024-03-23T07:56:22.097004+0000","last_peered":"2024-03-23T07:56:22.097004+0000","last_clean":"2024-03-23T07:56:22.097004+0000","last_became_active":"2024-03-23T07:56:17.731600+0000","last_became_peered":"2024-03-23T07:56:17.731600+0000","last_unstale":"2024-03-23T07:56:22.097004+0000","last_undegraded":"2024-03-23T07:56:22.097004+0000","last_fullsized":"2024-03-23T07:56:22.097004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:55.441592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988753+0000","last_change":"2024-03-23T07:56:17.724024+0000","last_active":"2024-03-23T07:56:21.988753+0000","last_peered":"2024-03-23T07:56:21.988753+0000","last_clean":"2024-03-23T07:56:21.988753+0000","last_became_active":"2024-03-23T07:56:17.723743+0000","last_became_peered":"2024-03-23T07:56:17.723743+0000","last_unstale":"2024-03-23T07:56:21.988753+0000","last_undegraded":"2024-03-23T07:56:21.988753+0000","last_fullsized":"2024-03-23T07:56:21.988753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:32:51.077761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002205+0000","last_change":"2024-03-23T07:56:17.723640+0000","last_active":"2024-03-23T07:56:22.002205+0000","last_peered":"2024-03-23T07:56:22.002205+0000","last_clean":"2024-03-23T07:56:22.002205+0000","last_became_active":"2024-03-23T07:56:17.723352+0000","last_became_peered":"2024-03-23T07:56:17.723352+0000","last_unstale":"2024-03-23T07:56:22.002205+0000","last_undegraded":"2024-03-23T07:56:22.002205+0000","last_fullsized":"2024-03-23T07:56:22.002205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:48.788785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002849+0000","last_change":"2024-03-23T07:56:17.745108+0000","last_active":"2024-03-23T07:56:22.002849+0000","last_peered":"2024-03-23T07:56:22.002849+0000","last_clean":"2024-03-23T07:56:22.002849+0000","last_became_active":"2024-03-23T07:56:17.744769+0000","last_became_peered":"2024-03-23T07:56:17.744769+0000","last_unstale":"2024-03-23T07:56:22.002849+0000","last_undegraded":"2024-03-23T07:56:22.002849+0000","last_fullsized":"2024-03-23T07:56:22.002849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:45.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096478+0000","last_change":"2024-03-23T07:56:17.743989+0000","last_active":"2024-03-23T07:56:22.096478+0000","last_peered":"2024-03-23T07:56:22.096478+0000","last_clean":"2024-03-23T07:56:22.096478+0000","last_became_active":"2024-03-23T07:56:17.743697+0000","last_became_peered":"2024-03-23T07:56:17.743697+0000","last_unstale":"2024-03-23T07:56:22.096478+0000","last_undegraded":"2024-03-23T07:56:22.096478+0000","last_fullsized":"2024-03-23T07:56:22.096478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:33:48.226604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096538+0000","last_change":"2024-03-23T07:56:17.737898+0000","last_active":"2024-03-23T07:56:22.096538+0000","last_peered":"2024-03-23T07:56:22.096538+0000","last_clean":"2024-03-23T07:56:22.096538+0000","last_became_active":"2024-03-23T07:56:17.737650+0000","last_became_peered":"2024-03-23T07:56:17.737650+0000","last_unstale":"2024-03-23T07:56:22.096538+0000","last_undegraded":"2024-03-23T07:56:22.096538+0000","last_fullsized":"2024-03-23T07:56:22.096538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:21.794118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096613+0000","last_change":"2024-03-23T07:56:17.741344+0000","last_active":"2024-03-23T07:56:22.096613+0000","last_peered":"2024-03-23T07:56:22.096613+0000","last_clean":"2024-03-23T07:56:22.096613+0000","last_became_active":"2024-03-23T07:56:17.740336+0000","last_became_peered":"2024-03-23T07:56:17.740336+0000","last_unstale":"2024-03-23T07:56:22.096613+0000","last_undegraded":"2024-03-23T07:56:22.096613+0000","last_fullsized":"2024-03-23T07:56:22.096613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093627+0000","last_change":"2024-03-23T07:56:17.725091+0000","last_active":"2024-03-23T07:56:22.093627+0000","last_peered":"2024-03-23T07:56:22.093627+0000","last_clean":"2024-03-23T07:56:22.093627+0000","last_became_active":"2024-03-23T07:56:17.724794+0000","last_became_peered":"2024-03-23T07:56:17.724794+0000","last_unstale":"2024-03-23T07:56:22.093627+0000","last_undegraded":"2024-03-23T07:56:22.093627+0000","last_fullsized":"2024-03-23T07:56:22.093627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983064+0000","last_change":"2024-03-23T07:56:17.749204+0000","last_active":"2024-03-23T07:56:21.983064+0000","last_peered":"2024-03-23T07:56:21.983064+0000","last_clean":"2024-03-23T07:56:21.983064+0000","last_became_active":"2024-03-23T07:56:17.748888+0000","last_became_peered":"2024-03-23T07:56:17.748888+0000","last_unstale":"2024-03-23T07:56:21.983064+0000","last_undegraded":"2024-03-23T07:56:21.983064+0000","last_fullsized":"2024-03-23T07:56:21.983064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:14.142988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096378+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096378+0000","last_peered":"2024-03-23T07:56:22.096378+0000","last_clean":"2024-03-23T07:56:22.096378+0000","last_became_active":"2024-03-23T07:56:17.743764+0000","last_became_peered":"2024-03-23T07:56:17.743764+0000","last_unstale":"2024-03-23T07:56:22.096378+0000","last_undegraded":"2024-03-23T07:56:22.096378+0000","last_fullsized":"2024-03-23T07:56:22.096378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093513+0000","last_change":"2024-03-23T07:56:17.724709+0000","last_active":"2024-03-23T07:56:22.093513+0000","last_peered":"2024-03-23T07:56:22.093513+0000","last_clean":"2024-03-23T07:56:22.093513+0000","last_became_active":"2024-03-23T07:56:17.724393+0000","last_became_peered":"2024-03-23T07:56:17.724393+0000","last_unstale":"2024-03-23T07:56:22.093513+0000","last_undegraded":"2024-03-23T07:56:22.093513+0000","last_fullsized":"2024-03-23T07:56:22.093513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096409+0000","last_change":"2024-03-23T07:56:17.742970+0000","last_active":"2024-03-23T07:56:22.096409+0000","last_peered":"2024-03-23T07:56:22.096409+0000","last_clean":"2024-03-23T07:56:22.096409+0000","last_became_active":"2024-03-23T07:56:17.742629+0000","last_became_peered":"2024-03-23T07:56:17.742629+0000","last_unstale":"2024-03-23T07:56:22.096409+0000","last_undegraded":"2024-03-23T07:56:22.096409+0000","last_fullsized":"2024-03-23T07:56:22.096409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:21:07.705635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093633+0000","last_change":"2024-03-23T07:56:17.736446+0000","last_active":"2024-03-23T07:56:22.093633+0000","last_peered":"2024-03-23T07:56:22.093633+0000","last_clean":"2024-03-23T07:56:22.093633+0000","last_became_active":"2024-03-23T07:56:17.727700+0000","last_became_peered":"2024-03-23T07:56:17.727700+0000","last_unstale":"2024-03-23T07:56:22.093633+0000","last_undegraded":"2024-03-23T07:56:22.093633+0000","last_fullsized":"2024-03-23T07:56:22.093633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991994+0000","last_change":"2024-03-23T07:56:17.732549+0000","last_active":"2024-03-23T07:56:21.991994+0000","last_peered":"2024-03-23T07:56:21.991994+0000","last_clean":"2024-03-23T07:56:21.991994+0000","last_became_active":"2024-03-23T07:56:17.732276+0000","last_became_peered":"2024-03-23T07:56:17.732276+0000","last_unstale":"2024-03-23T07:56:21.991994+0000","last_undegraded":"2024-03-23T07:56:21.991994+0000","last_fullsized":"2024-03-23T07:56:21.991994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:48.012231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000782+0000","last_change":"2024-03-23T07:56:17.732034+0000","last_active":"2024-03-23T07:56:22.000782+0000","last_peered":"2024-03-23T07:56:22.000782+0000","last_clean":"2024-03-23T07:56:22.000782+0000","last_became_active":"2024-03-23T07:56:17.731730+0000","last_became_peered":"2024-03-23T07:56:17.731730+0000","last_unstale":"2024-03-23T07:56:22.000782+0000","last_undegraded":"2024-03-23T07:56:22.000782+0000","last_fullsized":"2024-03-23T07:56:22.000782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:59:20.863874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096639+0000","last_change":"2024-03-23T07:56:17.746986+0000","last_active":"2024-03-23T07:56:22.096639+0000","last_peered":"2024-03-23T07:56:22.096639+0000","last_clean":"2024-03-23T07:56:22.096639+0000","last_became_active":"2024-03-23T07:56:17.746621+0000","last_became_peered":"2024-03-23T07:56:17.746621+0000","last_unstale":"2024-03-23T07:56:22.096639+0000","last_undegraded":"2024-03-23T07:56:22.096639+0000","last_fullsized":"2024-03-23T07:56:22.096639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:22.719041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476745,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]}]},{"osd":2,"up_from":16,"seq":68719476745,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]}]},{"osd":1,"up_from":16,"seq":68719476745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":694448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]}]},{"osd":0,"up_from":16,"seq":68719476747,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.871}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6599999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":108477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:56:47.050 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T07:56:47.050 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T07:56:47.050 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-23T07:56:47.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-23T07:56:47.051 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-23T07:56:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.296+0000 7f1d9fa00640 1 Processor -- start 2024-03-23T07:56:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.296+0000 7f1d9fa00640 1 -- start start 2024-03-23T07:56:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.298+0000 7f1d9fa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d980adbc0 0x7f1d980b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.298+0000 7f1d9fa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d9806d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.298+0000 7f1d9fa00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d980b16a0 con 0x7f1d980cbab0 2024-03-23T07:56:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.298+0000 7f1d9fa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9806dd40 con 0x7f1d980adbc0 2024-03-23T07:56:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.298+0000 7f1d9fa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980b4700 con 0x7f1d9806d390 2024-03-23T07:56:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.299+0000 7f1d9d000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d980adbc0 0x7f1d980b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.299+0000 7f1d9d000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d980adbc0 0x7f1d980b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:37898/0 (socket says 172.21.15.105:37898) 2024-03-23T07:56:47.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.299+0000 7f1d9d000640 1 -- 172.21.15.105:0/903199957 learned_addr learned my addr 172.21.15.105:0/903199957 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:47.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.299+0000 7f1d9da00640 1 --2- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d9806d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.300+0000 7f1d9d000640 1 -- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 msgr2=0x7f1d9806d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:47.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.300+0000 7f1d9d000640 1 --2- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d9806d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.300+0000 7f1d9d000640 1 -- 172.21.15.105:0/903199957 >> v1:172.21.15.132:6789/0 conn(0x7f1d980cbab0 legacy=0x7f1d980b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.300+0000 7f1d9d000640 1 -- 172.21.15.105:0/903199957 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980cbfb0 con 0x7f1d980adbc0 2024-03-23T07:56:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.301+0000 7f1d9d000640 1 --2- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980adbc0 0x7f1d980b41c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009870 tx=0x7f1d8802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7020526c7387141f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:47.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.301+0000 7f1d97400640 1 -- 172.21.15.105:0/903199957 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88031640 con 0x7f1d980adbc0 2024-03-23T07:56:47.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d97400640 1 -- 172.21.15.105:0/903199957 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d880317a0 con 0x7f1d980adbc0 2024-03-23T07:56:47.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d97400640 1 -- 172.21.15.105:0/903199957 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88031b30 con 0x7f1d980adbc0 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 -- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980adbc0 msgr2=0x7f1d980b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980adbc0 0x7f1d980b41c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009870 tx=0x7f1d8802b8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 -- 172.21.15.105:0/903199957 shutdown_connections 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d9806d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/903199957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980adbc0 0x7f1d980b41c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 -- 172.21.15.105:0/903199957 >> 172.21.15.105:0/903199957 conn(0x7f1d9806a3d0 msgr2=0x7f1d980b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 -- 172.21.15.105:0/903199957 shutdown_connections 2024-03-23T07:56:47.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.302+0000 7f1d9fa00640 1 -- 172.21.15.105:0/903199957 wait complete. 2024-03-23T07:56:47.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.304+0000 7f1d9fa00640 1 Processor -- start 2024-03-23T07:56:47.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.304+0000 7f1d9fa00640 1 -- start start 2024-03-23T07:56:47.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.305+0000 7f1d9fa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d98138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.305+0000 7f1d9fa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980adbc0 0x7f1d980afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.305+0000 7f1d9fa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980cbab0 0x7f1d980b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.305+0000 7f1d9fa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d98130d70 con 0x7f1d980adbc0 2024-03-23T07:56:47.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.305+0000 7f1d9fa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d98132cd0 con 0x7f1d980cbab0 2024-03-23T07:56:47.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.305+0000 7f1d9fa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d98131010 con 0x7f1d9806d390 2024-03-23T07:56:47.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.305+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980adbc0 0x7f1d980afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980adbc0 0x7f1d980afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33214/0 (socket says 172.21.15.105:33214) 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 -- 172.21.15.105:0/534098924 learned_addr learned my addr 172.21.15.105:0/534098924 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d9d000640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d98138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 -- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 msgr2=0x7f1d98138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d98138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 -- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980cbab0 msgr2=0x7f1d980b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d9da00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980cbab0 0x7f1d980b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980cbab0 0x7f1d980b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 -- 172.21.15.105:0/534098924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d880094d0 con 0x7f1d980adbc0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.306+0000 7f1d97e00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980adbc0 0x7f1d980afeb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c400 tx=0x7f1d8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.307+0000 7f1d94c00640 1 -- 172.21.15.105:0/534098924 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800052d0 con 0x7f1d980adbc0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.307+0000 7f1d94c00640 1 -- 172.21.15.105:0/534098924 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d80005430 con 0x7f1d980adbc0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.307+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d981312f0 con 0x7f1d980adbc0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.307+0000 7f1d94c00640 1 -- 172.21.15.105:0/534098924 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80003830 con 0x7f1d980adbc0 2024-03-23T07:56:47.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.307+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d981318b0 con 0x7f1d980adbc0 2024-03-23T07:56:47.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.308+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9806ac20 con 0x7f1d980adbc0 2024-03-23T07:56:47.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.309+0000 7f1d94c00640 1 -- 172.21.15.105:0/534098924 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1d80003990 con 0x7f1d980adbc0 2024-03-23T07:56:47.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.310+0000 7f1d94c00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d7406ac00 0x7f1d7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.310+0000 7f1d94c00640 1 -- 172.21.15.105:0/534098924 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1d80088cb0 con 0x7f1d980adbc0 2024-03-23T07:56:47.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.310+0000 7f1d9d000640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d7406ac00 0x7f1d7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.311+0000 7f1d9d000640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d7406ac00 0x7f1d7406d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d88036000 tx=0x7f1d8803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:47.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.314+0000 7f1d94c00640 1 -- 172.21.15.105:0/534098924 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1d80053b00 con 0x7f1d980adbc0 2024-03-23T07:56:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.555+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1d98073390 con 0x7f1d980adbc0 2024-03-23T07:56:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.560+0000 7f1d94c00640 1 -- 172.21.15.105:0/534098924 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170932 (secure 0 0 0) 0x7f1d80017090 con 0x7f1d980adbc0 2024-03-23T07:56:47.564 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d7406ac00 msgr2=0x7f1d7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d7406ac00 0x7f1d7406d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d88036000 tx=0x7f1d8803c040 comp rx=0 tx=0).stop 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980adbc0 msgr2=0x7f1d980afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980adbc0 0x7f1d980afeb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c400 tx=0x7f1d8000c8c0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 shutdown_connections 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d980cbab0 0x7f1d980b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d980adbc0 0x7f1d980afeb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d7406ac00 0x7f1d7406d0c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 --2- 172.21.15.105:0/534098924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9806d390 0x7f1d98138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 >> 172.21.15.105:0/534098924 conn(0x7f1d9806a3d0 msgr2=0x7f1d980ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 shutdown_connections 2024-03-23T07:56:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.565+0000 7f1d9fa00640 1 -- 172.21.15.105:0/534098924 wait complete. 2024-03-23T07:56:47.583 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":37091},{"type":"v1","addr":"172.21.15.105:6833","nonce":37091}]},"active_addr":"172.21.15.105:6833/37091","active_change":"2024-03-23T07:55:58.021742+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2440615759}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":795642450}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":377313989}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2257034007}]}]} 2024-03-23T07:56:47.587 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-23T07:56:47.588 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-23T07:56:47.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.784+0000 7f6161000640 1 Processor -- start 2024-03-23T07:56:47.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.784+0000 7f6161000640 1 -- start start 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.786+0000 7f6161000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615c0b5a30 0x7f615c0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.786+0000 7f6161000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c0d1830 con 0x7f615c15cda0 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.786+0000 7f6161000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c0b1f20 con 0x7f615c0b4350 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.786+0000 7f6161000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c0d0c20 con 0x7f615c0b5a30 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.786+0000 7f6159e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f615c15cda0 0x7f615c0cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59376/0 (socket says 172.21.15.105:59376) 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.786+0000 7f6159e00640 1 -- 172.21.15.105:0/2176351571 learned_addr learned my addr 172.21.15.105:0/2176351571 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6151400640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615c0b5a30 0x7f615c0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477892574 0 0) 0x7f615c0d1830 con 0x7f615c15cda0 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6140003520 con 0x7f615c15cda0 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2645191527 0 0) 0x7f6140003520 con 0x7f615c15cda0 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615c0b5a30 msgr2=0x7f615c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615c0b5a30 0x7f615c0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 >> v1:172.21.15.132:6789/0 conn(0x7f615c0b4350 legacy=0x7f615c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c0d0f00 con 0x7f615c15cda0 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f614c001670 con 0x7f615c15cda0 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 >> v1:172.21.15.105:6789/0 conn(0x7f615c15cda0 legacy=0x7f615c0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61400039a0 0x7f6140003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61400047c0 0x7f6140004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6140005180 0x7f614000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c0d0c20 con 0x7f6140005180 2024-03-23T07:56:47.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614000c050 con 0x7f61400047c0 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614000c210 con 0x7f61400039a0 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.787+0000 7f6151400640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f615c0b5a30 0x7f615c0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f615a800640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6140005180 0x7f614000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f6159e00640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61400039a0 0x7f6140003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f6151400640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61400047c0 0x7f6140004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f615a800640 1 -- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61400039a0 msgr2=0x7f6140003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f615a800640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61400039a0 0x7f6140003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f615a800640 1 -- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61400047c0 msgr2=0x7f6140004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f615a800640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61400047c0 0x7f6140004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.788+0000 7f615a800640 1 -- 172.21.15.105:0/2176351571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6140003520 con 0x7f6140005180 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.789+0000 7f615a800640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6140005180 0x7f614000b980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6154001e50 tx=0x7f61400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e338a311ac7b5d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154003b70 con 0x7f6140005180 2024-03-23T07:56:47.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154003cd0 con 0x7f6140005180 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6159400640 1 -- 172.21.15.105:0/2176351571 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154005310 con 0x7f6140005180 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6161000640 1 -- 172.21.15.105:0/2176351571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6140005180 msgr2=0x7f614000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6161000640 1 --2- 172.21.15.105:0/2176351571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6140005180 0x7f614000b980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6154001e50 tx=0x7f61400057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f615a800640 1 -- 172.21.15.105:0/2176351571 reap_dead start 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6161000640 1 -- 172.21.15.105:0/2176351571 shutdown_connections 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6161000640 1 -- 172.21.15.105:0/2176351571 >> 172.21.15.105:0/2176351571 conn(0x7f615c0cf350 msgr2=0x7f615c0cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6161000640 1 -- 172.21.15.105:0/2176351571 shutdown_connections 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.790+0000 7f6161000640 1 -- 172.21.15.105:0/2176351571 wait complete. 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.792+0000 7f6161000640 1 Processor -- start 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.792+0000 7f6161000640 1 -- start start 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.793+0000 7f6161000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6140005180 0x7f615c0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.793+0000 7f6161000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f615c1f60e0 0x7f615c0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.793+0000 7f6161000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f615c1f6560 0x7f615c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.793+0000 7f6161000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c1f5cc0 con 0x7f615c1f6560 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.793+0000 7f6161000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c1f5970 con 0x7f615c1f60e0 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.793+0000 7f6161000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c1f2ba0 con 0x7f6140005180 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f615c1f6560 0x7f615c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f615c1f6560 0x7f615c0cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33236/0 (socket says 172.21.15.105:33236) 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 -- 172.21.15.105:0/3610010004 learned_addr learned my addr 172.21.15.105:0/3610010004 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f6151400640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f615c1f60e0 0x7f615c0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f6159e00640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6140005180 0x7f615c0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 -- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6140005180 msgr2=0x7f615c0c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6140005180 0x7f615c0c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 -- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f615c1f60e0 msgr2=0x7f615c0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:47.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f615c1f60e0 0x7f615c0ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 -- 172.21.15.105:0/3610010004 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c1f2e80 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.794+0000 7f615a800640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f615c1f6560 0x7f615c0cd440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6154001f40 tx=0x7f61540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.795+0000 7f6152a00640 1 -- 172.21.15.105:0/3610010004 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615403e070 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.795+0000 7f6152a00640 1 -- 172.21.15.105:0/3610010004 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154003e40 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.795+0000 7f6152a00640 1 -- 172.21.15.105:0/3610010004 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154006d50 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.795+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c1f3160 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.795+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c0c65e0 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.796+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c1f3350 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.797+0000 7f6152a00640 1 -- 172.21.15.105:0/3610010004 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6154041070 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.798+0000 7f6152a00640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f613806ac00 0x7f613806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.798+0000 7f6152a00640 1 -- 172.21.15.105:0/3610010004 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61540aa650 con 0x7f615c1f6560 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.798+0000 7f6159e00640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f613806ac00 0x7f613806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.799+0000 7f6159e00640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f613806ac00 0x7f613806d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f614c000c00 tx=0x7f614c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:47.806+0000 7f6152a00640 1 -- 172.21.15.105:0/3610010004 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6154075520 con 0x7f615c1f6560 2024-03-23T07:56:48.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.006+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f615c073390 con 0x7f615c1f6560 2024-03-23T07:56:48.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.008+0000 7f6152a00640 1 -- 172.21.15.105:0/3610010004 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7f6154078b40 con 0x7f615c1f6560 2024-03-23T07:56:48.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:48.009 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f613806ac00 msgr2=0x7f613806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f613806ac00 0x7f613806d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f614c000c00 tx=0x7f614c009040 comp rx=0 tx=0).stop 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f615c1f6560 msgr2=0x7f615c0cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f615c1f6560 0x7f615c0cd440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6154001f40 tx=0x7f61540014a0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 shutdown_connections 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f615c1f6560 0x7f615c0cd440 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f615c1f60e0 0x7f615c0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f613806ac00 0x7f613806d0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 --2- 172.21.15.105:0/3610010004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6140005180 0x7f615c0c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 >> 172.21.15.105:0/3610010004 conn(0x7f615c0cf350 msgr2=0x7f615c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 shutdown_connections 2024-03-23T07:56:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.011+0000 7f6161000640 1 -- 172.21.15.105:0/3610010004 wait complete. 2024-03-23T07:56:48.029 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-23T07:56:48.029 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.218+0000 7f8ed1a00640 1 Processor -- start 2024-03-23T07:56:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.218+0000 7f8ed1a00640 1 -- start start 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ed1a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ecc15b560 0x7f8ecc15b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ed1a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc0b39b0 0x7f8ecc0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ed1a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc06c6f0 con 0x7f8ecc15e500 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ed1a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b4370 con 0x7f8ecc15b560 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ed1a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc06c9d0 con 0x7f8ecc0b39b0 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ecaa00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ecc15b560 0x7f8ecc15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ecaa00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ecc15b560 0x7f8ecc15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33238/0 (socket says 172.21.15.105:33238) 2024-03-23T07:56:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8ecaa00640 1 -- 172.21.15.105:0/3999296701 learned_addr learned my addr 172.21.15.105:0/3999296701 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.220+0000 7f8eca000640 1 --2- 172.21.15.105:0/3999296701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc0b39b0 0x7f8ecc0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.221+0000 7f8ecaa00640 1 -- 172.21.15.105:0/3999296701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc0b39b0 msgr2=0x7f8ecc0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.221+0000 7f8ecaa00640 1 --2- 172.21.15.105:0/3999296701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc0b39b0 0x7f8ecc0b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.221+0000 7f8ecaa00640 1 -- 172.21.15.105:0/3999296701 >> v1:172.21.15.132:6789/0 conn(0x7f8ecc15e500 legacy=0x7f8ecc15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.221+0000 7f8ecaa00640 1 -- 172.21.15.105:0/3999296701 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc06ccb0 con 0x7f8ecc15b560 2024-03-23T07:56:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.222+0000 7f8ecaa00640 1 --2- 172.21.15.105:0/3999296701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc15b940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009870 tx=0x7f8ebc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b82d1c8252afdcd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.222+0000 7f8ec9600640 1 -- 172.21.15.105:0/3999296701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc031640 con 0x7f8ecc15b560 2024-03-23T07:56:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.222+0000 7f8ec9600640 1 -- 172.21.15.105:0/3999296701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ebc0317a0 con 0x7f8ecc15b560 2024-03-23T07:56:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.222+0000 7f8ec9600640 1 -- 172.21.15.105:0/3999296701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc031b30 con 0x7f8ecc15b560 2024-03-23T07:56:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 -- 172.21.15.105:0/3999296701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 msgr2=0x7f8ecc15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/3999296701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc15b940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009870 tx=0x7f8ebc02b8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 -- 172.21.15.105:0/3999296701 shutdown_connections 2024-03-23T07:56:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/3999296701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc0b39b0 0x7f8ecc0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/3999296701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc15b940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 -- 172.21.15.105:0/3999296701 >> 172.21.15.105:0/3999296701 conn(0x7f8ecc06a3d0 msgr2=0x7f8ecc0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 -- 172.21.15.105:0/3999296701 shutdown_connections 2024-03-23T07:56:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.223+0000 7f8ed1a00640 1 -- 172.21.15.105:0/3999296701 wait complete. 2024-03-23T07:56:48.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.225+0000 7f8ed1a00640 1 Processor -- start 2024-03-23T07:56:48.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.225+0000 7f8ed1a00640 1 -- start start 2024-03-23T07:56:48.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8ed1a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ecc0b39b0 0x7f8ecc155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8ed1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc13fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8ed1a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc15e500 0x7f8ecc0abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8ed1a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b1af0 con 0x7f8ecc15b560 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8ed1a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b17a0 con 0x7f8ecc0b39b0 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8ed1a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc140100 con 0x7f8ecc15e500 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8eca000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc13fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8eca000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc13fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33254/0 (socket says 172.21.15.105:33254) 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8eca000640 1 -- 172.21.15.105:0/478788586 learned_addr learned my addr 172.21.15.105:0/478788586 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8eca000640 1 -- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc15e500 msgr2=0x7f8ecc0abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8eca000640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc15e500 0x7f8ecc0abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8eca000640 1 -- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ecc0b39b0 msgr2=0x7f8ecc155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8ecaa00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ecc0b39b0 0x7f8ecc155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.226+0000 7f8eca000640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ecc0b39b0 0x7f8ecc155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8eca000640 1 -- 172.21.15.105:0/478788586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc0094d0 con 0x7f8ecc15b560 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8ecaa00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ecc0b39b0 0x7f8ecc155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8eca000640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc13fbc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800d800 tx=0x7f8eb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8ec2a00640 1 -- 172.21.15.105:0/478788586 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800b6c0 con 0x7f8ecc15b560 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8ec2a00640 1 -- 172.21.15.105:0/478788586 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb800b820 con 0x7f8ecc15b560 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8ec2a00640 1 -- 172.21.15.105:0/478788586 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800bbb0 con 0x7f8ecc15b560 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc0ac510 con 0x7f8ecc15b560 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.227+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc0acad0 con 0x7f8ecc15b560 2024-03-23T07:56:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.228+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc0b1c30 con 0x7f8ecc15b560 2024-03-23T07:56:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.229+0000 7f8ec2a00640 1 -- 172.21.15.105:0/478788586 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8eb800bd10 con 0x7f8ecc15b560 2024-03-23T07:56:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.230+0000 7f8ec2a00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8eac06ac00 0x7f8eac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.230+0000 7f8ec2a00640 1 -- 172.21.15.105:0/478788586 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8eb8089c40 con 0x7f8ecc15b560 2024-03-23T07:56:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.230+0000 7f8ecaa00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8eac06ac00 0x7f8eac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.231+0000 7f8ecaa00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8eac06ac00 0x7f8eac06d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc036000 tx=0x7f8ebc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.235+0000 7f8ec2a00640 1 -- 172.21.15.105:0/478788586 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8ecc0b1c30 con 0x7f8ecc15b560 2024-03-23T07:56:48.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.433+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ecc0b1c30 con 0x7f8ecc15b560 2024-03-23T07:56:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.436+0000 7f8ec2a00640 1 -- 172.21.15.105:0/478788586 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7f8ecc0b1c30 con 0x7f8ecc15b560 2024-03-23T07:56:48.437 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:48.437 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:48.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8eac06ac00 msgr2=0x7f8eac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8eac06ac00 0x7f8eac06d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc036000 tx=0x7f8ebc03c040 comp rx=0 tx=0).stop 2024-03-23T07:56:48.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 msgr2=0x7f8ecc13fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc13fbc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800d800 tx=0x7f8eb800dcc0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 shutdown_connections 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8eac06ac00 0x7f8eac06d0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ecc15e500 0x7f8ecc0abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc15b560 0x7f8ecc13fbc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 --2- 172.21.15.105:0/478788586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ecc0b39b0 0x7f8ecc155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.438+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 >> 172.21.15.105:0/478788586 conn(0x7f8ecc06a3d0 msgr2=0x7f8ecc0ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.439+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 shutdown_connections 2024-03-23T07:56:48.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.439+0000 7f8ed1a00640 1 -- 172.21.15.105:0/478788586 wait complete. 2024-03-23T07:56:48.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-23T07:56:48.457 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-23T07:56:48.457 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-23T07:56:48.458 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-23T07:56:48.458 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-23T07:56:48.458 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-23T07:56:48.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-23T07:56:48.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-23T07:56:48.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.760+0000 7f260fe00640 1 Processor -- start 2024-03-23T07:56:48.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.760+0000 7f260fe00640 1 -- start start 2024-03-23T07:56:48.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.764+0000 7f72ce400640 1 Processor -- start 2024-03-23T07:56:48.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.765+0000 7f72ce400640 1 -- start start 2024-03-23T07:56:48.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.762+0000 7f260fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26100b16a0 0x7f26100cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.762+0000 7f260fe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2610169790 0x7f26100b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.763+0000 7f260fe00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26100b1fc0 con 0x7f26100b2790 2024-03-23T07:56:48.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.763+0000 7f260fe00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2610169c70 con 0x7f2610169790 2024-03-23T07:56:48.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.763+0000 7f260fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2610169f50 con 0x7f26100b16a0 2024-03-23T07:56:48.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.763+0000 7f260ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26100b16a0 0x7f26100cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.763+0000 7f260f400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f26100b2790 0x7f261016d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59382/0 (socket says 172.21.15.105:59382) 2024-03-23T07:56:48.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.763+0000 7f260f400640 1 -- 172.21.15.105:0/977913914 learned_addr learned my addr 172.21.15.105:0/977913914 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:48.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.766+0000 7f72ce400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 0x7f72c80ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.767+0000 7f72ce400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72c80b30e0 0x7f72c80b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.767+0000 7f72ce400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c815fa50 con 0x7f72c8070170 2024-03-23T07:56:48.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.767+0000 7f72ce400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c806d2c0 con 0x7f72c80b30e0 2024-03-23T07:56:48.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.767+0000 7f72ce400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80b3a80 con 0x7f72c8156790 2024-03-23T07:56:48.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.768+0000 7f72c7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 0x7f72c80ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.769+0000 7f72c7e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f72c8070170 0x7f72c815c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59392/0 (socket says 172.21.15.105:59392) 2024-03-23T07:56:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.769+0000 7f72c7e00640 1 -- 172.21.15.105:0/1694431332 learned_addr learned my addr 172.21.15.105:0/1694431332 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:48.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.769+0000 7f72c6a00640 1 --2- 172.21.15.105:0/1694431332 >> v2:172.21.15.105:3300/0 conn(0x7f72c80b30e0 0x7f72c80b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.766+0000 7f260ea00640 1 -- 172.21.15.105:0/977913914 >> v2:172.21.15.105:3300/0 conn(0x7f2610169790 msgr2=0x7f26100b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.766+0000 7f260ea00640 1 --2- 172.21.15.105:0/977913914 >> v2:172.21.15.105:3300/0 conn(0x7f2610169790 0x7f26100b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.766+0000 7f260ea00640 1 -- 172.21.15.105:0/977913914 >> v1:172.21.15.105:6789/0 conn(0x7f26100b2790 legacy=0x7f261016d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.766+0000 7f260ea00640 1 -- 172.21.15.105:0/977913914 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261016a230 con 0x7f26100b16a0 2024-03-23T07:56:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.771+0000 7f72c6000640 1 -- 172.21.15.105:0/1694431332 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149528920 0 0) 0x7f72c815fa50 con 0x7f72c8070170 2024-03-23T07:56:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.771+0000 7f72c6000640 1 -- 172.21.15.105:0/1694431332 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72ac003520 con 0x7f72c8070170 2024-03-23T07:56:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.771+0000 7f72c6a00640 1 -- 172.21.15.105:0/1694431332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 msgr2=0x7f72c80ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.771+0000 7f72c6a00640 1 --2- 172.21.15.105:0/1694431332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 0x7f72c80ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.771+0000 7f72c6000640 1 -- 172.21.15.105:0/1694431332 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1398578480 0 0) 0x7f72ac003520 con 0x7f72c8070170 2024-03-23T07:56:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.771+0000 7f72c6a00640 1 -- 172.21.15.105:0/1694431332 >> v1:172.21.15.105:6789/0 conn(0x7f72c8070170 legacy=0x7f72c815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.771+0000 7f72c6a00640 1 -- 172.21.15.105:0/1694431332 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c806d6f0 con 0x7f72c80b30e0 2024-03-23T07:56:48.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.773+0000 7f260ea00640 1 --2- 172.21.15.105:0/977913914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26100b16a0 0x7f26100cce10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2610071320 tx=0x7f25fc02cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc9ca7ed2f39e1d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.774+0000 7f72c6a00640 1 --2- 172.21.15.105:0/1694431332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c80b30e0 0x7f72c80b3540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009a00 tx=0x7f72b402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d466e4c00da7b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.774+0000 7f2b13a00640 1 Processor -- start 2024-03-23T07:56:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.774+0000 7f2b13a00640 1 -- start start 2024-03-23T07:56:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.774+0000 7f72c6000640 1 -- 172.21.15.105:0/1694431332 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c00019d0 con 0x7f72c80b30e0 2024-03-23T07:56:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.775+0000 7f72c6000640 1 -- 172.21.15.105:0/1694431332 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b4030440 con 0x7f72c80b30e0 2024-03-23T07:56:48.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.775+0000 7f72c6000640 1 -- 172.21.15.105:0/1694431332 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b40307b0 con 0x7f72c80b30e0 2024-03-23T07:56:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.775+0000 7f72ce400640 1 -- 172.21.15.105:0/1694431332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c80b30e0 msgr2=0x7f72c80b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.775+0000 7f72ce400640 1 --2- 172.21.15.105:0/1694431332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c80b30e0 0x7f72c80b3540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009a00 tx=0x7f72b402ba10 comp rx=0 tx=0).stop 2024-03-23T07:56:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f72ce400640 1 -- 172.21.15.105:0/1694431332 shutdown_connections 2024-03-23T07:56:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f72ce400640 1 --2- 172.21.15.105:0/1694431332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c80b30e0 0x7f72c80b3540 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f72ce400640 1 --2- 172.21.15.105:0/1694431332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 0x7f72c80ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f72ce400640 1 -- 172.21.15.105:0/1694431332 >> 172.21.15.105:0/1694431332 conn(0x7f72c806a0b0 msgr2=0x7f72c806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f72ce400640 1 -- 172.21.15.105:0/1694431332 shutdown_connections 2024-03-23T07:56:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f72ce400640 1 -- 172.21.15.105:0/1694431332 wait complete. 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f2b13a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cd6b0 0x7f2b0c0b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f2b0be00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2b0c15cd90 0x7f2b0c06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59402/0 (socket says 172.21.15.105:59402) 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.777+0000 7f2b11000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cd6b0 0x7f2b0c0b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.777+0000 7f2b11000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cd6b0 0x7f2b0c0b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52142/0 (socket says 172.21.15.105:52142) 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.777+0000 7f2b11000640 1 -- 172.21.15.105:0/4247271675 learned_addr learned my addr 172.21.15.105:0/4247271675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.776+0000 7f2b13a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0c06b840 con 0x7f2b0c15cd90 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.777+0000 7f2b13a00640 1 -- 172.21.15.105:0/4247271675 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0c06bb20 con 0x7f2b0c0cda90 2024-03-23T07:56:48.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.777+0000 7f2b13a00640 1 -- 172.21.15.105:0/4247271675 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c06c0e0 con 0x7f2b0c0cd6b0 2024-03-23T07:56:48.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.778+0000 7f260d600640 1 -- 172.21.15.105:0/977913914 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc02d9f0 con 0x7f26100b16a0 2024-03-23T07:56:48.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.778+0000 7f260d600640 1 -- 172.21.15.105:0/977913914 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc02db50 con 0x7f26100b16a0 2024-03-23T07:56:48.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.778+0000 7f260d600640 1 -- 172.21.15.105:0/977913914 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc035440 con 0x7f26100b16a0 2024-03-23T07:56:48.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.778+0000 7f2b0b400640 1 -- 172.21.15.105:0/4247271675 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42506645 0 0) 0x7f2b0c06bb20 con 0x7f2b0c0cda90 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.778+0000 7f2b0b400640 1 -- 172.21.15.105:0/4247271675 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af0003520 con 0x7f2b0c0cda90 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.778+0000 7f2b11000640 1 -- 172.21.15.105:0/4247271675 >> v1:172.21.15.132:6789/0 conn(0x7f2b0c0cda90 legacy=0x7f2b0c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.779+0000 7f260fe00640 1 -- 172.21.15.105:0/977913914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26100b16a0 msgr2=0x7f26100cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.779+0000 7f260fe00640 1 --2- 172.21.15.105:0/977913914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26100b16a0 0x7f26100cce10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2610071320 tx=0x7f25fc02cc40 comp rx=0 tx=0).stop 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.779+0000 7f260fe00640 1 -- 172.21.15.105:0/977913914 shutdown_connections 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.779+0000 7f260fe00640 1 --2- 172.21.15.105:0/977913914 >> v2:172.21.15.105:3300/0 conn(0x7f2610169790 0x7f26100b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.779+0000 7f260fe00640 1 --2- 172.21.15.105:0/977913914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26100b16a0 0x7f26100cce10 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2610071320 tx=0x7f25fc02cc40 comp rx=0 tx=0).stop 2024-03-23T07:56:48.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.779+0000 7f260fe00640 1 -- 172.21.15.105:0/977913914 >> 172.21.15.105:0/977913914 conn(0x7f2610069e70 msgr2=0x7f261006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.780+0000 7f5552c00640 1 Processor -- start 2024-03-23T07:56:48.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.780+0000 7f2b11000640 1 -- 172.21.15.105:0/4247271675 >> v1:172.21.15.105:6789/0 conn(0x7f2b0c15cd90 legacy=0x7f2b0c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.780+0000 7f2b11000640 1 -- 172.21.15.105:0/4247271675 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c06be00 con 0x7f2b0c0cd6b0 2024-03-23T07:56:48.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.782+0000 7f2b11000640 1 --2- 172.21.15.105:0/4247271675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cd6b0 0x7f2b0c0b8490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0098a0 tx=0x7f2afc02ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779193f78ece2eef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.780+0000 7f5552c00640 1 -- start start 2024-03-23T07:56:48.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.782+0000 7f72ce400640 1 Processor -- start 2024-03-23T07:56:48.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.780+0000 7f260fe00640 1 -- 172.21.15.105:0/977913914 shutdown_connections 2024-03-23T07:56:48.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.782+0000 7f72ce400640 1 -- start start 2024-03-23T07:56:48.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.782+0000 7f2b0b400640 1 -- 172.21.15.105:0/4247271675 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc02f820 con 0x7f2b0c0cd6b0 2024-03-23T07:56:48.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.780+0000 7f260fe00640 1 -- 172.21.15.105:0/977913914 wait complete. 2024-03-23T07:56:48.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f260fe00640 1 Processor -- start 2024-03-23T07:56:48.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f5552c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f554c1627c0 0x7f554c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f5552c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c1657c0 con 0x7f554c165340 2024-03-23T07:56:48.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f5552c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c06cdc0 con 0x7f554c0ce930 2024-03-23T07:56:48.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f5552c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554c161ab0 con 0x7f554c1627c0 2024-03-23T07:56:48.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f2b0b400640 1 -- 172.21.15.105:0/4247271675 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc02f980 con 0x7f2b0c0cd6b0 2024-03-23T07:56:48.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f2b0b400640 1 -- 172.21.15.105:0/4247271675 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc02fd10 con 0x7f2b0c0cd6b0 2024-03-23T07:56:48.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.784+0000 7f554b400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f554c165340 0x7f554c0ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59408/0 (socket says 172.21.15.105:59408) 2024-03-23T07:56:48.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.784+0000 7f72ce400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c8070170 0x7f72c81409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.784+0000 7f554b400640 1 -- 172.21.15.105:0/1877545073 learned_addr learned my addr 172.21.15.105:0/1877545073 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:48.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.784+0000 7f72c7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c8070170 0x7f72c81409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.784+0000 7f72c7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c8070170 0x7f72c81409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33296/0 (socket says 172.21.15.105:33296) 2024-03-23T07:56:48.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f72c7400640 1 -- 172.21.15.105:0/42063583 learned_addr learned my addr 172.21.15.105:0/42063583 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f2b13a00640 1 -- 172.21.15.105:0/4247271675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cd6b0 msgr2=0x7f2b0c0b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f2b13a00640 1 --2- 172.21.15.105:0/4247271675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cd6b0 0x7f2b0c0b8490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0098a0 tx=0x7f2afc02ba00 comp rx=0 tx=0).stop 2024-03-23T07:56:48.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f72ce400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c80b30e0 0x7f72c815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f72ce400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 0x7f72c8148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8140ef0 con 0x7f72c8070170 2024-03-23T07:56:48.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8140520 con 0x7f72c80b30e0 2024-03-23T07:56:48.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8146110 con 0x7f72c8156790 2024-03-23T07:56:48.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f2b13a00640 1 -- 172.21.15.105:0/4247271675 shutdown_connections 2024-03-23T07:56:48.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f2b13a00640 1 --2- 172.21.15.105:0/4247271675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cd6b0 0x7f2b0c0b8490 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.784+0000 7f5550c00640 1 --2- 172.21.15.105:0/1877545073 >> v2:172.21.15.132:3300/0 conn(0x7f554c1627c0 0x7f554c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f2b13a00640 1 -- 172.21.15.105:0/4247271675 >> 172.21.15.105:0/4247271675 conn(0x7f2b0c0cf780 msgr2=0x7f2b0c0cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f72c6a00640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c80b30e0 0x7f72c815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f2b13a00640 1 -- 172.21.15.105:0/4247271675 shutdown_connections 2024-03-23T07:56:48.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.785+0000 7f2b13a00640 1 -- 172.21.15.105:0/4247271675 wait complete. 2024-03-23T07:56:48.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.783+0000 7f260fe00640 1 -- start start 2024-03-23T07:56:48.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.786+0000 7f72c7e00640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 0x7f72c8148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.786+0000 7f72c7400640 1 -- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 msgr2=0x7f72c8148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.786+0000 7f72c7400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c8156790 0x7f72c8148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.786+0000 7f72c7400640 1 -- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c80b30e0 msgr2=0x7f72c815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.786+0000 7f72c7400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c80b30e0 0x7f72c815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.786+0000 7f72c7400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4009400 con 0x7f72c8070170 2024-03-23T07:56:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7f72c7400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c8070170 0x7f72c81409b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e870 tx=0x7f72b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7f260fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26100b2790 0x7f26101642c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7f260fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2610169790 0x7f2610159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7fea8f400640 1 Processor -- start 2024-03-23T07:56:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984258983 0 0) 0x7f5538001110 con 0x7f554c165340 2024-03-23T07:56:48.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5530003520 con 0x7f554c165340 2024-03-23T07:56:48.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7fea8f400640 1 -- start start 2024-03-23T07:56:48.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.787+0000 7f260fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26100b0310 0x7f26101680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26100a7810 con 0x7f26100b0310 2024-03-23T07:56:48.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26100b2790 0x7f26101642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26100a7af0 con 0x7f2610169790 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26100a7dd0 con 0x7f26100b2790 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26100b2790 0x7f26101642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52170/0 (socket says 172.21.15.105:52170) 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260ea00640 1 -- 172.21.15.105:0/1193497058 learned_addr learned my addr 172.21.15.105:0/1193497058 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260f400640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26100b0310 0x7f26101680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f260e000640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2610169790 0x7f2610159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2768265559 0 0) 0x7f5530003520 con 0x7f554c165340 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 >> v2:172.21.15.132:3300/0 conn(0x7f554c1627c0 msgr2=0x7f554c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 --2- 172.21.15.105:0/1877545073 >> v2:172.21.15.132:3300/0 conn(0x7f554c1627c0 0x7f554c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 >> v1:172.21.15.132:6789/0 conn(0x7f554c0ce930 legacy=0x7f554c1612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c161d90 con 0x7f554c165340 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f5538001800 con 0x7f554c165340 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 >> v1:172.21.15.105:6789/0 conn(0x7f554c165340 legacy=0x7f554c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5530003970 0x7f5530003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55300047c0 0x7f5530004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5530005180 0x7f553000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c161ab0 con 0x7f5530005180 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553000c000 con 0x7f55300047c0 2024-03-23T07:56:48.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553000c1c0 con 0x7f5530003970 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b80094f0 con 0x7f72c8070170 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800c940 con 0x7f72c8070170 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800ccf0 con 0x7f72c8070170 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.788+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c813fbf0 con 0x7f72c8070170 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c81401b0 con 0x7f72c8070170 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f260e000640 1 -- 172.21.15.105:0/1193497058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26100b2790 msgr2=0x7f26101642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f260e000640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26100b2790 0x7f26101642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f260e000640 1 -- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26100b0310 msgr2=0x7f26101680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f260e000640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26100b0310 0x7f26101680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f260e000640 1 -- 172.21.15.105:0/1193497058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc009c90 con 0x7f2610169790 2024-03-23T07:56:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f554be00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5530003970 0x7f5530003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7f2b13a00640 1 Processor -- start 2024-03-23T07:56:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.789+0000 7fea8f400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fea880cd6b0 0x7fea880b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.790+0000 7f2b13a00640 1 -- start start 2024-03-23T07:56:48.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.790+0000 7fea8f400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8806b840 con 0x7fea8815cd90 2024-03-23T07:56:48.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.790+0000 7f5550c00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5530005180 0x7f553000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.791+0000 7f260e000640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2610169790 0x7f2610159c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2600009f30 tx=0x7f260000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.790+0000 7fea8f400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8806bb20 con 0x7fea880cda90 2024-03-23T07:56:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.791+0000 7fea87e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fea8815cd90 0x7fea8806b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59420/0 (socket says 172.21.15.105:59420) 2024-03-23T07:56:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.791+0000 7fea87e00640 1 -- 172.21.15.105:0/1078613934 learned_addr learned my addr 172.21.15.105:0/1078613934 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.791+0000 7fea8ca00640 1 --2- 172.21.15.105:0/1078613934 >> v2:172.21.15.132:3300/0 conn(0x7fea880cd6b0 0x7fea880b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.791+0000 7fea8f400640 1 -- 172.21.15.105:0/1078613934 --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8806c0e0 con 0x7fea880cd6b0 2024-03-23T07:56:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.791+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600010a30 con 0x7f2610169790 2024-03-23T07:56:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.791+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72a0000fc0 con 0x7f72c8070170 2024-03-23T07:56:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2b13a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b0c0cd6b0 0x7f2b0c15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2b13a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cda90 0x7f2b0c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2b13a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b0c15cd90 0x7f2b0c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2b13a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c0cb150 con 0x7f2b0c0cd6b0 2024-03-23T07:56:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2b13a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c0cb430 con 0x7f2b0c15cd90 2024-03-23T07:56:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2b13a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c0cb710 con 0x7f2b0c0cda90 2024-03-23T07:56:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f554be00640 1 -- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55300047c0 msgr2=0x7f5530004c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f554be00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55300047c0 0x7f5530004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f554be00640 1 -- 172.21.15.105:0/1877545073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5530005180 msgr2=0x7f553000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f554be00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5530005180 0x7f553000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f554be00640 1 -- 172.21.15.105:0/1877545073 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c161d90 con 0x7f5530003970 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f5550c00640 1 -- 172.21.15.105:0/1877545073 reap_dead start 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b0c15cd90 0x7f2b0c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b0c15cd90 0x7f2b0c0cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:37982/0 (socket says 172.21.15.105:37982) 2024-03-23T07:56:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 -- 172.21.15.105:0/368073303 learned_addr learned my addr 172.21.15.105:0/368073303 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7fea8ca00640 1 -- 172.21.15.105:0/1078613934 >> v1:172.21.15.132:6789/0 conn(0x7fea880cda90 legacy=0x7fea880d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7fea8ca00640 1 -- 172.21.15.105:0/1078613934 >> v1:172.21.15.105:6789/0 conn(0x7fea8815cd90 legacy=0x7fea8806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7fea8ca00640 1 -- 172.21.15.105:0/1078613934 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8806be00 con 0x7fea880cd6b0 2024-03-23T07:56:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26100a80b0 con 0x7f2610169790 2024-03-23T07:56:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f260f400640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26100b0310 0x7f26101680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600010b90 con 0x7f2610169790 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.792+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600018440 con 0x7f2610169790 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f260ea00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26100b2790 0x7f26101642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.794+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26100a74a0 con 0x7f2610169790 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 -- 172.21.15.105:0/368073303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cda90 msgr2=0x7f2b0c15dc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b0c0cda90 0x7f2b0c15dc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.794+0000 7f554be00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5530003970 0x7f5530003d70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5544001d50 tx=0x7f55300057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace88e087bea2682 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.794+0000 7fea8ca00640 1 --2- 172.21.15.105:0/1078613934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea880cd6b0 0x7fea880b8490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea740097a0 tx=0x7fea7402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220eb0eedcc9d6ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 -- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b0c0cd6b0 msgr2=0x7f2b0c15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b0c0cd6b0 0x7f2b0c15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.793+0000 7f2b11a00640 1 -- 172.21.15.105:0/368073303 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0094d0 con 0x7f2b0c15cd90 2024-03-23T07:56:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.794+0000 7f2b11a00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b0c15cd90 0x7f2b0c0cc1f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c400 tx=0x7f2af800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26100ca780 con 0x7f2610169790 2024-03-23T07:56:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544003990 con 0x7f5530003970 2024-03-23T07:56:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7fea87400640 1 -- 172.21.15.105:0/1078613934 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7402e070 con 0x7fea880cd6b0 2024-03-23T07:56:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af80052d0 con 0x7f2b0c15cd90 2024-03-23T07:56:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0c0cb9f0 con 0x7f2b0c15cd90 2024-03-23T07:56:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8005430 con 0x7f2b0c15cd90 2024-03-23T07:56:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af80037f0 con 0x7f2b0c15cd90 2024-03-23T07:56:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f72b8013470 con 0x7f72c8070170 2024-03-23T07:56:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7fea87400640 1 -- 172.21.15.105:0/1078613934 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740315a0 con 0x7fea880cd6b0 2024-03-23T07:56:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.796+0000 7fea87400640 1 -- 172.21.15.105:0/1078613934 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74031930 con 0x7fea880cd6b0 2024-03-23T07:56:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544003b10 con 0x7f5530003970 2024-03-23T07:56:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.795+0000 7f554aa00640 1 -- 172.21.15.105:0/1877545073 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544005070 con 0x7f5530003970 2024-03-23T07:56:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.796+0000 7f5552c00640 1 -- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5530003970 msgr2=0x7f5530003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.796+0000 7f5552c00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5530003970 0x7f5530003d70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5544001d50 tx=0x7f55300057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.796+0000 7f5552c00640 1 -- 172.21.15.105:0/1877545073 shutdown_connections 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.796+0000 7f5552c00640 1 --2- 172.21.15.105:0/1877545073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5530003970 0x7f5530003d70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.796+0000 7f5552c00640 1 -- 172.21.15.105:0/1877545073 >> 172.21.15.105:0/1877545073 conn(0x7f554c06a0b0 msgr2=0x7f554c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.796+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0c06b3d0 con 0x7f2b0c15cd90 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.797+0000 7fea8f400640 1 -- 172.21.15.105:0/1078613934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea880cd6b0 msgr2=0x7fea880b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.797+0000 7fea8f400640 1 --2- 172.21.15.105:0/1078613934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea880cd6b0 0x7fea880b8490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea740097a0 tx=0x7fea7402b840 comp rx=0 tx=0).stop 2024-03-23T07:56:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.797+0000 7fea8f400640 1 -- 172.21.15.105:0/1078613934 shutdown_connections 2024-03-23T07:56:48.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.797+0000 7f72bf400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72a806ac30 0x7f72a806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b0c0cb5a0 con 0x7f2b0c15cd90 2024-03-23T07:56:48.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f72b8089ac0 con 0x7f72c8070170 2024-03-23T07:56:48.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7f72bf400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f72a8074270 0x7f72a80766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.797+0000 7fea8f400640 1 --2- 172.21.15.105:0/1078613934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea880cd6b0 0x7fea880b8490 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.797+0000 7fea8f400640 1 -- 172.21.15.105:0/1078613934 >> 172.21.15.105:0/1078613934 conn(0x7fea880cf780 msgr2=0x7fea880cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7f5552c00640 1 -- 172.21.15.105:0/1877545073 shutdown_connections 2024-03-23T07:56:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7f5552c00640 1 -- 172.21.15.105:0/1877545073 wait complete. 2024-03-23T07:56:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f26000213b0 con 0x7f2610169790 2024-03-23T07:56:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.799+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72a8076d80 con 0x7f72a8074270 2024-03-23T07:56:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.799+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72b8089e70 con 0x7f72c8070170 2024-03-23T07:56:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.799+0000 7f72c7e00640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f72a8074270 0x7f72a80766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.800+0000 7f72c7e00640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f72a8074270 0x7f72a80766b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.800+0000 7f72c6a00640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72a806ac30 0x7f72a806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7fea8f400640 1 -- 172.21.15.105:0/1078613934 shutdown_connections 2024-03-23T07:56:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.798+0000 7fea8f400640 1 -- 172.21.15.105:0/1078613934 wait complete. 2024-03-23T07:56:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.801+0000 7fea8f400640 1 Processor -- start 2024-03-23T07:56:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.801+0000 7fea8f400640 1 -- start start 2024-03-23T07:56:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.802+0000 7f72c6a00640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72a806ac30 0x7f72a806d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72b4036000 tx=0x7f72b4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.802+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== osd.0 v2:172.21.15.105:6800/37342 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f72a8076d80 con 0x7f72a8074270 2024-03-23T07:56:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.801+0000 7f5552c00640 1 Processor -- start 2024-03-23T07:56:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.802+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2af8003950 con 0x7f2b0c15cd90 2024-03-23T07:56:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.802+0000 7f5552c00640 1 -- start start 2024-03-23T07:56:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.803+0000 7fea8f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea880cd6b0 0x7fea88147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.803+0000 7f2606a00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25f006ac30 0x7f25f006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.803+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f260008d840 con 0x7f2610169790 2024-03-23T07:56:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.803+0000 7f2606a00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f25f0074270 0x7f25f00766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 --> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25f0076d80 con 0x7f25f0074270 2024-03-23T07:56:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f260008dbf0 con 0x7f2610169790 2024-03-23T07:56:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea880cda90 0x7fea8815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7f5552c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5530003970 0x7f554c1601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8ca00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea880cd6b0 0x7fea88147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8ca00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea880cd6b0 0x7fea88147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33356/0 (socket says 172.21.15.105:33356) 2024-03-23T07:56:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8ca00640 1 -- 172.21.15.105:0/1841051347 learned_addr learned my addr 172.21.15.105:0/1841051347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8f400640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8815cd90 0x7fea880cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7f5552c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f554c0cc240 0x7f554c0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881484d0 con 0x7fea880cd6b0 2024-03-23T07:56:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c8c90 con 0x7fea8815cd90 2024-03-23T07:56:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8815fff0 con 0x7fea880cda90 2024-03-23T07:56:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7f260f400640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f25f0074270 0x7f25f00766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7f2b08c00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2aec06ac00 0x7f2aec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.804+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2af8088b50 con 0x7f2b0c15cd90 2024-03-23T07:56:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.805+0000 7f2b08c00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7f2aec074240 0x7f2aec076680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.805+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 --> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2aec076d50 con 0x7f2aec074240 2024-03-23T07:56:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.805+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2af808d2a0 con 0x7f2b0c15cd90 2024-03-23T07:56:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.805+0000 7f5552c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554c0cc6c0 0x7f554c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.805+0000 7f2b0be00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7f2aec074240 0x7f2aec076680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f260ea00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25f006ac30 0x7f25f006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.805+0000 7f554be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5530003970 0x7f554c1601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f5552c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0ae3a0 con 0x7f554c0cc6c0 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f5552c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c160720 con 0x7f5530003970 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f5552c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0ccc20 con 0x7f554c0cc240 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f554be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5530003970 0x7f554c1601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:37992/0 (socket says 172.21.15.105:37992) 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f554be00640 1 -- 172.21.15.105:0/1862283993 learned_addr learned my addr 172.21.15.105:0/1862283993 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f5550c00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554c0cc6c0 0x7f554c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.805+0000 7f554b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f554c0cc240 0x7f554c0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.806+0000 7f2b0be00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7f2aec074240 0x7f2aec076680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.807+0000 7f2b11000640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2aec06ac00 0x7f2aec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.807+0000 7fea87e00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea880cda90 0x7fea8815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.808+0000 7fea8ca00640 1 -- 172.21.15.105:0/1841051347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea880cda90 msgr2=0x7fea8815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.808+0000 7fea8ca00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea880cda90 0x7fea8815fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.808+0000 7fea8ca00640 1 -- 172.21.15.105:0/1841051347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8815cd90 msgr2=0x7fea880cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.808+0000 7fea8ca00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8815cd90 0x7fea880cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.808+0000 7fea8ca00640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea74009400 con 0x7fea880cd6b0 2024-03-23T07:56:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.808+0000 7f5550c00640 1 -- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f554c0cc240 msgr2=0x7f554c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.807+0000 7f4cf0a00640 1 Processor -- start 2024-03-23T07:56:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.807+0000 7f4cf0a00640 1 -- start start 2024-03-23T07:56:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== osd.2 v2:172.21.15.105:6808/37346 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2aec076d50 con 0x7f2aec074240 2024-03-23T07:56:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7fea8ca00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea880cd6b0 0x7fea88147a90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea740097a0 tx=0x7fea740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74032900 con 0x7fea880cd6b0 2024-03-23T07:56:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f5550c00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f554c0cc240 0x7f554c0b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f5550c00640 1 -- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5530003970 msgr2=0x7f554c1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f5550c00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5530003970 0x7f554c1601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f5550c00640 1 -- 172.21.15.105:0/1862283993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0ccf00 con 0x7f554c0cc6c0 2024-03-23T07:56:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f554be00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5530003970 0x7f554c1601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f260f400640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f25f0074270 0x7f25f00766b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f260ea00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25f006ac30 0x7f25f006d0f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25fc02cc80 tx=0x7f25fc031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8815ea10 con 0x7fea880cd6b0 2024-03-23T07:56:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.810+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea74032a60 con 0x7fea880cd6b0 2024-03-23T07:56:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8815efd0 con 0x7fea880cd6b0 2024-03-23T07:56:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.809+0000 7f554b400640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f554c0cc240 0x7f554c0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.810+0000 7f5550c00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554c0cc6c0 0x7f554c0b1490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f554c162ba0 tx=0x7f553c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.811+0000 7f2b11000640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2aec06ac00 0x7f2aec06d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2afc036000 tx=0x7f2afc039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.811+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74032900 con 0x7fea880cd6b0 2024-03-23T07:56:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.811+0000 7f4cf0a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cec070730 0x7f4cec159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.812+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== osd.1 v2:172.21.15.105:6812/37345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f25f0076d80 con 0x7f25f0074270 2024-03-23T07:56:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.812+0000 7f4cf0a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4cec06d410 0x7f4cec06d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.812+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea880cdf40 con 0x7fea880cd6b0 2024-03-23T07:56:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.812+0000 7f4cf0a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec06cf50 con 0x7f4cec158990 2024-03-23T07:56:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.812+0000 7f4cf0a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cec159db0 con 0x7f4cec06d410 2024-03-23T07:56:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.812+0000 7f4cf0a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cec15c7b0 con 0x7f4cec070730 2024-03-23T07:56:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.814+0000 7f4ce9e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cec070730 0x7f4cec159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.814+0000 7f4ce9e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cec070730 0x7f4cec159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38004/0 (socket says 172.21.15.105:38004) 2024-03-23T07:56:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.814+0000 7f4ce9e00640 1 -- 172.21.15.105:0/1164932529 learned_addr learned my addr 172.21.15.105:0/1164932529 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.814+0000 7f4ce9e00640 1 -- 172.21.15.105:0/1164932529 >> v2:172.21.15.105:3300/0 conn(0x7f4cec06d410 msgr2=0x7f4cec06d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.814+0000 7f4ce9e00640 1 --2- 172.21.15.105:0/1164932529 >> v2:172.21.15.105:3300/0 conn(0x7f4cec06d410 0x7f4cec06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.814+0000 7f4ce9e00640 1 -- 172.21.15.105:0/1164932529 >> v1:172.21.15.105:6789/0 conn(0x7f4cec158990 legacy=0x7f4cec070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.815+0000 7f4ce9e00640 1 -- 172.21.15.105:0/1164932529 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec15ca90 con 0x7f4cec070730 2024-03-23T07:56:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.815+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c004500 con 0x7f554c0cc6c0 2024-03-23T07:56:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.815+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c003610 con 0x7f554c0cc6c0 2024-03-23T07:56:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.815+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c0039c0 con 0x7f554c0cc6c0 2024-03-23T07:56:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.815+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c0cd1e0 con 0x7f554c0cc6c0 2024-03-23T07:56:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.816+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c1f6600 con 0x7f554c0cc6c0 2024-03-23T07:56:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.817+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f554c06d010 con 0x7f554c0cc6c0 2024-03-23T07:56:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.816+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fea74032bd0 con 0x7fea880cd6b0 2024-03-23T07:56:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.818+0000 7f4ce9e00640 1 --2- 172.21.15.105:0/1164932529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec159870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4cec15d4a0 tx=0x7f4cd802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998076341ba171ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.818+0000 7fea84c00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fea6806ac60 0x7fea6806d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.818+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea740ab3d0 con 0x7fea880cd6b0 2024-03-23T07:56:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.819+0000 7fea84c00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7fea680742a0 0x7fea680766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.819+0000 7fea87e00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fea6806ac60 0x7fea6806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.819+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea68076db0 con 0x7fea680742a0 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.819+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea740ab6f0 con 0x7fea880cd6b0 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.820+0000 7fea8d400640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7fea680742a0 0x7fea680766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.820+0000 7fea87e00640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fea6806ac60 0x7fea6806d120 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea881e8a50 tx=0x7fea7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.820+0000 7f4ce9400640 1 -- 172.21.15.105:0/1164932529 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8030860 con 0x7f4cec070730 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.821+0000 7f4ce9400640 1 -- 172.21.15.105:0/1164932529 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd80309c0 con 0x7f4cec070730 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.821+0000 7f4ce9400640 1 -- 172.21.15.105:0/1164932529 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8030d50 con 0x7f4cec070730 2024-03-23T07:56:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.821+0000 7fea8d400640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7fea680742a0 0x7fea680766e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.822+0000 7f4cf0a00640 1 -- 172.21.15.105:0/1164932529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 msgr2=0x7f4cec159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.822+0000 7f4cf0a00640 1 --2- 172.21.15.105:0/1164932529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec159870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4cec15d4a0 tx=0x7f4cd802b9a0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.823+0000 7f4cf0a00640 1 -- 172.21.15.105:0/1164932529 shutdown_connections 2024-03-23T07:56:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.823+0000 7f4cf0a00640 1 --2- 172.21.15.105:0/1164932529 >> v2:172.21.15.105:3300/0 conn(0x7f4cec06d410 0x7f4cec06d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.823+0000 7f4cf0a00640 1 --2- 172.21.15.105:0/1164932529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec159870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.823+0000 7f4cf0a00640 1 -- 172.21.15.105:0/1164932529 >> 172.21.15.105:0/1164932529 conn(0x7f4cec069e70 msgr2=0x7f4cec06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.823+0000 7f4cf0a00640 1 -- 172.21.15.105:0/1164932529 shutdown_connections 2024-03-23T07:56:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.823+0000 7f4cf0a00640 1 -- 172.21.15.105:0/1164932529 wait complete. 2024-03-23T07:56:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.824+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== osd.3 v2:172.21.15.105:6824/37347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fea68076db0 con 0x7fea680742a0 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.826+0000 7f4cf0a00640 1 Processor -- start 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.827+0000 7f4cf0a00640 1 -- start start 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.824+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f553c003b20 con 0x7f554c0cc6c0 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.826+0000 7f5543e00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f552c06ac50 0x7f552c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.826+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f553c089e60 con 0x7f554c0cc6c0 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.826+0000 7f5543e00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f552c074290 0x7f552c0766d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.826+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 --> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f552c076da0 con 0x7f552c074290 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.826+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553c08a290 con 0x7f554c0cc6c0 2024-03-23T07:56:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.827+0000 7f554be00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f552c06ac50 0x7f552c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.827+0000 7f554b400640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f552c074290 0x7f552c0766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.828+0000 7f554be00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f552c06ac50 0x7f552c06d110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5544001d50 tx=0x7f5544009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.829+0000 7f554b400640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f552c074290 0x7f552c0766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.829+0000 7f4cf0a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cec06d410 0x7f4cec1ff780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.829+0000 7f4cf0a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec0a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.830+0000 7f4cf0a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cec158990 0x7f4cec0aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.830+0000 7f4cea800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cec06d410 0x7f4cec1ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.830+0000 7f4cea800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cec06d410 0x7f4cec1ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33376/0 (socket says 172.21.15.105:33376) 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.830+0000 7f4ce9e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec0a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f4ce9e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec0a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38012/0 (socket says 172.21.15.105:38012) 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f4ceb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cec158990 0x7f4cec0aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f4ceb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cec158990 0x7f4cec0aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52214/0 (socket says 172.21.15.105:52214) 2024-03-23T07:56:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f4cf0a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0b1100 con 0x7f4cec06d410 2024-03-23T07:56:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f4cf0a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec1ffcc0 con 0x7f4cec070730 2024-03-23T07:56:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f4cf0a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0aca60 con 0x7f4cec158990 2024-03-23T07:56:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f4cea800640 1 -- 172.21.15.105:0/809360723 learned_addr learned my addr 172.21.15.105:0/809360723 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 --> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f72a0002cb0 con 0x7f72a8074270 2024-03-23T07:56:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== osd.5 v2:172.21.15.132:6817/37443 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f552c076da0 con 0x7f552c074290 2024-03-23T07:56:48.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.831+0000 7fc728200640 1 Processor -- start 2024-03-23T07:56:48.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.832+0000 7fc728200640 1 -- start start 2024-03-23T07:56:48.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.832+0000 7f72bf400640 1 -- 172.21.15.105:0/42063583 <== osd.0 v2:172.21.15.105:6800/37342 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f72a0002cb0 con 0x7f72a8074270 2024-03-23T07:56:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.833+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f72a8074270 msgr2=0x7f72a80766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.833+0000 7f72ce400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f72a8074270 0x7f72a80766b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.833+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72a806ac30 msgr2=0x7f72a806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.833+0000 7f72ce400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72a806ac30 0x7f72a806d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72b4036000 tx=0x7f72b4008030 comp rx=0 tx=0).stop 2024-03-23T07:56:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c8070170 msgr2=0x7f72c81409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7f72ce400640 1 --2- 172.21.15.105:0/42063583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72c8070170 0x7f72c81409b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e870 tx=0x7f72b800ed30 comp rx=0 tx=0).stop 2024-03-23T07:56:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7f72c7e00640 1 -- 172.21.15.105:0/42063583 reap_dead start 2024-03-23T07:56:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 shutdown_connections 2024-03-23T07:56:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 >> 172.21.15.105:0/42063583 conn(0x7f72c806a0b0 msgr2=0x7f72c80b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7f4cea800640 1 -- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cec158990 msgr2=0x7f4cec0aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7f4cea800640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cec158990 0x7f4cec0aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7f4cea800640 1 -- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 msgr2=0x7f4cec0a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7f4cea800640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec0a9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7f4cea800640 1 -- 172.21.15.105:0/809360723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd80094d0 con 0x7f4cec06d410 2024-03-23T07:56:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7f4ce9e00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cec070730 0x7f4cec0a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7f4ceb200640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cec158990 0x7f4cec0aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7fc728200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc7200b1f20 0x7fc7200b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7fc728200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7200b50c0 con 0x7fc7200b4c90 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7fc728200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7201607d0 con 0x7fc72015cc70 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.834+0000 7fc728200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc720160a70 con 0x7fc7200b1f20 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7fc725800640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc72015cc70 0x7fc7200cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.105:45520/0 (socket says 172.21.15.105:45520) 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7fc725800640 1 -- 172.21.15.105:0/573230145 learned_addr learned my addr 172.21.15.105:0/573230145 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.835+0000 7fc724e00640 1 --2- 172.21.15.105:0/573230145 >> v2:172.21.15.105:3300/0 conn(0x7fc7200b1f20 0x7fc7200b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248971992 0 0) 0x7fc7201607d0 con 0x7fc72015cc70 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc708003520 con 0x7fc72015cc70 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119471187 0 0) 0x7fc7200b50c0 con 0x7fc7200b4c90 2024-03-23T07:56:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc708003840 con 0x7fc7200b4c90 2024-03-23T07:56:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 shutdown_connections 2024-03-23T07:56:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7f72ce400640 1 -- 172.21.15.105:0/42063583 wait complete. 2024-03-23T07:56:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7f4cea800640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cec06d410 0x7f4cec1ff780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c3c0 tx=0x7f4cdc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 384145062 0 0) 0x7fc708003520 con 0x7fc72015cc70 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 >> v2:172.21.15.105:3300/0 conn(0x7fc7200b1f20 msgr2=0x7fc7200b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 --2- 172.21.15.105:0/573230145 >> v2:172.21.15.105:3300/0 conn(0x7fc7200b1f20 0x7fc7200b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 >> v1:172.21.15.105:6789/0 conn(0x7fc7200b4c90 legacy=0x7fc7200d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.836+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720160d50 con 0x7fc72015cc70 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fc714001650 con 0x7fc72015cc70 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 >> v1:172.21.15.132:6789/0 conn(0x7fc72015cc70 legacy=0x7fc7200cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc708004110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7080047c0 0x7fc708004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc708005270 0x7fc70800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720160a70 con 0x7fc708005270 2024-03-23T07:56:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70800c250 con 0x7fc7080047c0 2024-03-23T07:56:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.837+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70800c410 con 0x7fc708003cd0 2024-03-23T07:56:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.838+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0052d0 con 0x7f4cec06d410 2024-03-23T07:56:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.838+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc005450 con 0x7f4cec06d410 2024-03-23T07:56:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.838+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007260 con 0x7f4cec06d410 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.838+0000 7fc725800640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc708004110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.838+0000 7fc724e00640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7080047c0 0x7fc708004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7fc726200640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc708005270 0x7fc70800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7fc725800640 1 -- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7080047c0 msgr2=0x7fc708004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7fc725800640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7080047c0 0x7fc708004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7fc725800640 1 -- 172.21.15.105:0/573230145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc708005270 msgr2=0x7fc70800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7fc725800640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc708005270 0x7fc70800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7fc725800640 1 -- 172.21.15.105:0/573230145 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720160d50 con 0x7fc708003cd0 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.840+0000 7fc726200640 1 -- 172.21.15.105:0/573230145 reap_dead start 2024-03-23T07:56:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec0acd40 con 0x7f4cec06d410 2024-03-23T07:56:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.839+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec0ad300 con 0x7f4cec06d410 2024-03-23T07:56:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.841+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 --> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f261016a7f0 con 0x7f25f0074270 2024-03-23T07:56:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.841+0000 7fc725800640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc708004110 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc714013a40 tx=0x7fc7080057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622b2b9ef7cea5e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.842+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140357f0 con 0x7fc708003cd0 2024-03-23T07:56:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.842+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc714035950 con 0x7fc708003cd0 2024-03-23T07:56:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.842+0000 7fc71fe00640 1 -- 172.21.15.105:0/573230145 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714035d80 con 0x7fc708003cd0 2024-03-23T07:56:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.842+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cc4000fc0 con 0x7f4cec06d410 2024-03-23T07:56:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.843+0000 7f2606a00640 1 -- 172.21.15.105:0/1193497058 <== osd.1 v2:172.21.15.105:6812/37345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2608001d10 con 0x7f25f0074270 2024-03-23T07:56:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.843+0000 7fc728200640 1 -- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 msgr2=0x7fc708004110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.843+0000 7fc728200640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc708004110 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc714013a40 tx=0x7fc7080057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.843+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 --> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b0c0cb880 con 0x7f2aec074240 2024-03-23T07:56:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7fc728200640 1 -- 172.21.15.105:0/573230145 shutdown_connections 2024-03-23T07:56:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f25f0074270 msgr2=0x7f25f00766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7f260fe00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f25f0074270 0x7f25f00766b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7fc728200640 1 --2- 172.21.15.105:0/573230145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc708004110 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7fc728200640 1 -- 172.21.15.105:0/573230145 >> 172.21.15.105:0/573230145 conn(0x7fc7200cf790 msgr2=0x7fc7200cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7fc728200640 1 -- 172.21.15.105:0/573230145 shutdown_connections 2024-03-23T07:56:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7fc728200640 1 -- 172.21.15.105:0/573230145 wait complete. 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25f006ac30 msgr2=0x7f25f006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7f260fe00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25f006ac30 0x7f25f006d0f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25fc02cc80 tx=0x7f25fc031000 comp rx=0 tx=0).stop 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2610169790 msgr2=0x7f2610159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.844+0000 7f260fe00640 1 --2- 172.21.15.105:0/1193497058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2610169790 0x7f2610159c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2600009f30 tx=0x7f260000b870 comp rx=0 tx=0).stop 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.845+0000 7f260f400640 1 -- 172.21.15.105:0/1193497058 reap_dead start 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.845+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 shutdown_connections 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.845+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 >> 172.21.15.105:0/1193497058 conn(0x7f2610069e70 msgr2=0x7f261006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.845+0000 7f2b08c00640 1 -- 172.21.15.105:0/368073303 <== osd.2 v2:172.21.15.105:6808/37346 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2b0c0cb880 con 0x7f2aec074240 2024-03-23T07:56:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.846+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 shutdown_connections 2024-03-23T07:56:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.846+0000 7f260fe00640 1 -- 172.21.15.105:0/1193497058 wait complete. 2024-03-23T07:56:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.847+0000 7fc728200640 1 Processor -- start 2024-03-23T07:56:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.847+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7f2aec074240 msgr2=0x7f2aec076680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.847+0000 7fc728200640 1 -- start start 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.847+0000 7f2b13a00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7f2aec074240 0x7f2aec076680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2aec06ac00 msgr2=0x7f2aec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f2b13a00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2aec06ac00 0x7f2aec06d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2afc036000 tx=0x7f2afc039040 comp rx=0 tx=0).stop 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b0c15cd90 msgr2=0x7f2b0c0cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f2b13a00640 1 --2- 172.21.15.105:0/368073303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b0c15cd90 0x7f2b0c0cc1f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c400 tx=0x7f2af800c8c0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.846+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4cdc0073c0 con 0x7f4cec06d410 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f4ce2a00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4ccc06ac50 0x7f4ccc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4cdc0888b0 con 0x7f4cec06d410 2024-03-23T07:56:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f4ce2a00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f4ccc074210 0x7f4ccc076650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.848+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 --> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ccc076d20 con 0x7f4ccc074210 2024-03-23T07:56:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.849+0000 7f2b11a00640 1 -- 172.21.15.105:0/368073303 reap_dead start 2024-03-23T07:56:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.849+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cdc0538c0 con 0x7f4cec06d410 2024-03-23T07:56:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.849+0000 7fc728200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc720155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.849+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 shutdown_connections 2024-03-23T07:56:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.849+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 >> 172.21.15.105:0/368073303 conn(0x7f2b0c0cf780 msgr2=0x7f2b0c0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.849+0000 7fc728200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc72015fea0 0x7fc720154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.849+0000 7fc728200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7200cbb90 0x7fc7200cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc728200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200cc550 con 0x7fc72015fea0 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc725800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc720155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc725800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc720155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52246/0 (socket says 172.21.15.105:52246) 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc725800640 1 -- 172.21.15.105:0/631974093 learned_addr learned my addr 172.21.15.105:0/631974093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200c77e0 con 0x7fc7200cbb90 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72006cf50 con 0x7fc708003cd0 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 shutdown_connections 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc724e00640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc72015fea0 0x7fc720154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7fc726200640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7200cbb90 0x7fc7200cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7f2b13a00640 1 -- 172.21.15.105:0/368073303 wait complete. 2024-03-23T07:56:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7f4ce9e00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4ccc06ac50 0x7f4ccc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.850+0000 7f4ceb200640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f4ccc074210 0x7f4ccc076650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.851+0000 7fc724e00640 1 -- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 msgr2=0x7fc720155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.851+0000 7fc724e00640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc708003cd0 0x7fc720155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.851+0000 7fc724e00640 1 -- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7200cbb90 msgr2=0x7fc7200cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.851+0000 7fc724e00640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7200cbb90 0x7fc7200cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.851+0000 7fc724e00640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72015e280 con 0x7fc72015fea0 2024-03-23T07:56:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.852+0000 7fc724e00640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc72015fea0 0x7fc720154c60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7200b2320 tx=0x7fc70c010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.852+0000 7f4ceb200640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f4ccc074210 0x7f4ccc076650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.853+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00e8f0 con 0x7fc72015fea0 2024-03-23T07:56:48.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.853+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72015e560 con 0x7fc72015fea0 2024-03-23T07:56:48.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.853+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc720160320 con 0x7fc72015fea0 2024-03-23T07:56:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.854+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c00ea50 con 0x7fc72015fea0 2024-03-23T07:56:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.854+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00e8f0 con 0x7fc72015fea0 2024-03-23T07:56:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.854+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6fc000fc0 con 0x7fc72015fea0 2024-03-23T07:56:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.855+0000 7f4ce9e00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4ccc06ac50 0x7f4ccc06d110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8036000 tx=0x7f4cd8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.856+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== osd.7 v2:172.21.15.132:6816/37452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4ccc076d20 con 0x7f4ccc074210 2024-03-23T07:56:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.858+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc70c00ebc0 con 0x7fc72015fea0 2024-03-23T07:56:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.859+0000 7fc71d600640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc70006ac90 0x7fc70006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.860+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc70c058620 con 0x7fc72015fea0 2024-03-23T07:56:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.860+0000 7fc725800640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc70006ac90 0x7fc70006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.860+0000 7fc71d600640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7fc700074f70 0x7fc700077410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.860+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc700077ae0 con 0x7fc700074f70 2024-03-23T07:56:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.860+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70c0583f0 con 0x7fc72015fea0 2024-03-23T07:56:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.862+0000 7fc726200640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7fc700074f70 0x7fc700077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.864+0000 7fc725800640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc70006ac90 0x7fc70006d150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc714000c00 tx=0x7fc714038000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.865+0000 7fc726200640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7fc700074f70 0x7fc700077410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.867 INFO:teuthology.orchestra.run.smithi105.stdout:68719476749 2024-03-23T07:56:48.868 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-23T07:56:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.867+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== osd.4 v2:172.21.15.132:6800/37441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc700077ae0 con 0x7fc700074f70 2024-03-23T07:56:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.867+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 --> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f554c073390 con 0x7f552c074290 2024-03-23T07:56:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.867+0000 7fb4e1400640 1 Processor -- start 2024-03-23T07:56:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.868+0000 7fb4e1400640 1 -- start start 2024-03-23T07:56:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.868+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 --> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea88073390 con 0x7fea680742a0 2024-03-23T07:56:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.869+0000 7f5543e00640 1 -- 172.21.15.105:0/1862283993 <== osd.5 v2:172.21.15.132:6817/37443 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f554c073390 con 0x7f552c074290 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.872+0000 7fea84c00640 1 -- 172.21.15.105:0/1841051347 <== osd.3 v2:172.21.15.105:6824/37347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea88073390 con 0x7fea680742a0 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7fea680742a0 msgr2=0x7fea680766e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fea8f400640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7fea680742a0 0x7fea680766e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fea6806ac60 msgr2=0x7fea6806d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fea8f400640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fea6806ac60 0x7fea6806d120 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea881e8a50 tx=0x7fea7c008040 comp rx=0 tx=0).stop 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea880cd6b0 msgr2=0x7fea88147a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fea8f400640 1 --2- 172.21.15.105:0/1841051347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea880cd6b0 0x7fea88147a90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea740097a0 tx=0x7fea740046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fea8d400640 1 -- 172.21.15.105:0/1841051347 reap_dead start 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.870+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f552c074290 msgr2=0x7f552c0766d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.870+0000 7f5552c00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f552c074290 0x7f552c0766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.872+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f552c06ac50 msgr2=0x7f552c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.872+0000 7f5552c00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f552c06ac50 0x7f552c06d110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5544001d50 tx=0x7f5544009030 comp rx=0 tx=0).stop 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554c0cc6c0 msgr2=0x7f554c0b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7f5552c00640 1 --2- 172.21.15.105:0/1862283993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554c0cc6c0 0x7f554c0b1490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f554c162ba0 tx=0x7f553c011920 comp rx=0 tx=0).stop 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7f5550c00640 1 -- 172.21.15.105:0/1862283993 reap_dead start 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 shutdown_connections 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 >> 172.21.15.105:0/1862283993 conn(0x7f554c06a0b0 msgr2=0x7f554c163790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.874+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 shutdown_connections 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.871+0000 7fb4e1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.871+0000 7fb4e1400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb4dc15cc00 0x7fb4dc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.871+0000 7fb4e1400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc15d220 con 0x7fb4dc158c90 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.871+0000 7fb4e1400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc0b25d0 con 0x7fb4dc15cc00 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.871+0000 7fb4e1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc15d030 con 0x7fb4dc1588b0 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.872+0000 7fb4da800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.872+0000 7fb4d9e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb4dc158c90 0x7fb4dc15c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59442/0 (socket says 172.21.15.105:59442) 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.872+0000 7fb4d9e00640 1 -- 172.21.15.105:0/3744258458 learned_addr learned my addr 172.21.15.105:0/3744258458 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fb4d9400640 1 -- 172.21.15.105:0/3744258458 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539296157 0 0) 0x7fb4dc15d220 con 0x7fb4dc158c90 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fb4d9400640 1 -- 172.21.15.105:0/3744258458 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4c0003520 con 0x7fb4dc158c90 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fb4da800640 1 -- 172.21.15.105:0/3744258458 >> v2:172.21.15.132:3300/0 conn(0x7fb4dc15cc00 msgr2=0x7fb4dc0706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fb4da800640 1 --2- 172.21.15.105:0/3744258458 >> v2:172.21.15.132:3300/0 conn(0x7fb4dc15cc00 0x7fb4dc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fb4da800640 1 -- 172.21.15.105:0/3744258458 >> v1:172.21.15.105:6789/0 conn(0x7fb4dc158c90 legacy=0x7fb4dc15c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.873+0000 7fb4da800640 1 -- 172.21.15.105:0/3744258458 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc1591c0 con 0x7fb4dc1588b0 2024-03-23T07:56:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.874+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 shutdown_connections 2024-03-23T07:56:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.874+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 >> 172.21.15.105:0/1841051347 conn(0x7fea880cf780 msgr2=0x7fea880b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.874+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 shutdown_connections 2024-03-23T07:56:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.874+0000 7fea8f400640 1 -- 172.21.15.105:0/1841051347 wait complete. 2024-03-23T07:56:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.875+0000 7f5552c00640 1 -- 172.21.15.105:0/1862283993 wait complete. 2024-03-23T07:56:48.879 INFO:teuthology.orchestra.run.smithi105.stdout:68719476747 2024-03-23T07:56:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.878+0000 7fb4da800640 1 --2- 172.21.15.105:0/3744258458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc070170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097a0 tx=0x7fb4cc02b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e533d293c7727f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.880 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-23T07:56:48.881 INFO:teuthology.orchestra.run.smithi105.stdout:68719476747 2024-03-23T07:56:48.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.878+0000 7fb4d9400640 1 -- 172.21.15.105:0/3744258458 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc031710 con 0x7fb4dc1588b0 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.878+0000 7fb4d9400640 1 -- 172.21.15.105:0/3744258458 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc02f050 con 0x7fb4dc1588b0 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.878+0000 7fb4d9400640 1 -- 172.21.15.105:0/3744258458 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc031ad0 con 0x7fb4dc1588b0 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.880+0000 7fb4e1400640 1 -- 172.21.15.105:0/3744258458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 msgr2=0x7fb4dc070170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.880+0000 7fb4e1400640 1 --2- 172.21.15.105:0/3744258458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc070170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097a0 tx=0x7fb4cc02b7b0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.880+0000 7fb4e1400640 1 -- 172.21.15.105:0/3744258458 shutdown_connections 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.880+0000 7fb4e1400640 1 --2- 172.21.15.105:0/3744258458 >> v2:172.21.15.132:3300/0 conn(0x7fb4dc15cc00 0x7fb4dc0706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.880+0000 7fb4e1400640 1 --2- 172.21.15.105:0/3744258458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc070170 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.880+0000 7fb4e1400640 1 -- 172.21.15.105:0/3744258458 >> 172.21.15.105:0/3744258458 conn(0x7fb4dc069c90 msgr2=0x7fb4dc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.881+0000 7fb4e1400640 1 -- 172.21.15.105:0/3744258458 shutdown_connections 2024-03-23T07:56:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.881+0000 7fb4e1400640 1 -- 172.21.15.105:0/3744258458 wait complete. 2024-03-23T07:56:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.884+0000 7fb4e1400640 1 Processor -- start 2024-03-23T07:56:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.885+0000 7fb4e1400640 1 -- start start 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.888+0000 7fb4e1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.888+0000 7fb4e1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4dc158c90 0x7fb4dc1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.888+0000 7fb4e1400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4dc15cc00 0x7fb4dc0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.889+0000 7fb4db200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4dc15cc00 0x7fb4dc0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.889+0000 7fb4db200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4dc15cc00 0x7fb4dc0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33400/0 (socket says 172.21.15.105:33400) 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.889+0000 7fb4db200640 1 -- 172.21.15.105:0/535434322 learned_addr learned my addr 172.21.15.105:0/535434322 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.889+0000 7fb4e1400640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0af450 con 0x7fb4dc15cc00 2024-03-23T07:56:48.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.889+0000 7fb4e1400640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0ae7d0 con 0x7fb4dc158c90 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.889+0000 7fb4e1400640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0aeab0 con 0x7fb4dc1588b0 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.889+0000 7fb4da800640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.890+0000 7fb4d9e00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4dc158c90 0x7fb4dc1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.890+0000 7fb4db200640 1 -- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 msgr2=0x7fb4dc142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.890+0000 7fb4db200640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4dc1588b0 0x7fb4dc142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.890+0000 7fb4db200640 1 -- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4dc158c90 msgr2=0x7fb4dc1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.890+0000 7fb4db200640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4dc158c90 0x7fb4dc1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.890+0000 7fb4db200640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc009400 con 0x7fb4dc15cc00 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.890+0000 7fb4db200640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4dc15cc00 0x7fb4dc0cc730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400ff70 tx=0x7fb4d4010550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.891+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400d0d0 con 0x7fb4dc15cc00 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.891+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d400d250 con 0x7fb4dc15cc00 2024-03-23T07:56:48.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.891+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400d600 con 0x7fb4dc15cc00 2024-03-23T07:56:48.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.892+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 --> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4cc4002cb0 con 0x7f4ccc074210 2024-03-23T07:56:48.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.891+0000 7fb4e1400640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc159e20 con 0x7fb4dc15cc00 2024-03-23T07:56:48.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.892+0000 7fb4e1400640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc0af070 con 0x7fb4dc15cc00 2024-03-23T07:56:48.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.893+0000 7f4ce2a00640 1 -- 172.21.15.105:0/809360723 <== osd.7 v2:172.21.15.132:6816/37452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4cc4002cb0 con 0x7f4ccc074210 2024-03-23T07:56:48.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.894+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f4ccc074210 msgr2=0x7f4ccc076650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.895+0000 7f4cf0a00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f4ccc074210 0x7f4ccc076650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.895+0000 7fb4e1400640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4b4000fc0 con 0x7fb4dc15cc00 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.895+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4ccc06ac50 msgr2=0x7f4ccc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.895+0000 7f4cf0a00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4ccc06ac50 0x7f4ccc06d110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8036000 tx=0x7f4cd8008030 comp rx=0 tx=0).stop 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.895+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cec06d410 msgr2=0x7f4cec1ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.895+0000 7f4cf0a00640 1 --2- 172.21.15.105:0/809360723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cec06d410 0x7f4cec1ff780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c3c0 tx=0x7f4cdc00c880 comp rx=0 tx=0).stop 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.896+0000 7f4ceb200640 1 -- 172.21.15.105:0/809360723 reap_dead start 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.896+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 shutdown_connections 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.896+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 >> 172.21.15.105:0/809360723 conn(0x7f4cec069e70 msgr2=0x7f4cec15b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.896+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 shutdown_connections 2024-03-23T07:56:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.896+0000 7f4cf0a00640 1 -- 172.21.15.105:0/809360723 wait complete. 2024-03-23T07:56:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.897+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb4d400b780 con 0x7fb4dc15cc00 2024-03-23T07:56:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.899+0000 7fb4d2a00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb4bc06ac30 0x7fb4bc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.899+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb4d408d440 con 0x7fb4dc15cc00 2024-03-23T07:56:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.900+0000 7fb4d2a00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7fb4bc0741f0 0x7fb4bc076630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.900+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4bc076d00 con 0x7fb4bc0741f0 2024-03-23T07:56:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.900+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d40580c0 con 0x7fb4dc15cc00 2024-03-23T07:56:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.901+0000 7fb4da800640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb4bc06ac30 0x7fb4bc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.903 INFO:teuthology.orchestra.run.smithi105.stdout:60129542154 2024-03-23T07:56:48.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-23T07:56:48.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.903+0000 7fb4da800640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb4bc06ac30 0x7fb4bc06d0f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc035000 tx=0x7fb4cc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.904+0000 7fb4d9e00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7fb4bc0741f0 0x7fb4bc076630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:48.910 INFO:teuthology.orchestra.run.smithi105.stdout:68719476747 2024-03-23T07:56:48.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.907+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 --> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc6fc002cb0 con 0x7fc700074f70 2024-03-23T07:56:48.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.909+0000 7fc71d600640 1 -- 172.21.15.105:0/631974093 <== osd.4 v2:172.21.15.132:6800/37441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc6fc002cb0 con 0x7fc700074f70 2024-03-23T07:56:48.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.905+0000 7fb4d9e00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7fb4bc0741f0 0x7fb4bc076630 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:48.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.907+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== osd.6 v2:172.21.15.132:6808/37442 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb4bc076d00 con 0x7fb4bc0741f0 2024-03-23T07:56:48.911 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-23T07:56:48.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.909+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7fc700074f70 msgr2=0x7fc700077410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.909+0000 7fc728200640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7fc700074f70 0x7fc700077410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.910+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc70006ac90 msgr2=0x7fc70006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.910+0000 7fc728200640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc70006ac90 0x7fc70006d150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc714000c00 tx=0x7fc714038000 comp rx=0 tx=0).stop 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.910+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc72015fea0 msgr2=0x7fc720154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.910+0000 7fc728200640 1 --2- 172.21.15.105:0/631974093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc72015fea0 0x7fc720154c60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7200b2320 tx=0x7fc70c010b70 comp rx=0 tx=0).stop 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.910+0000 7fc726200640 1 -- 172.21.15.105:0/631974093 reap_dead start 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.910+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 shutdown_connections 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.910+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 >> 172.21.15.105:0/631974093 conn(0x7fc7200cf790 msgr2=0x7fc72015bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.911+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 shutdown_connections 2024-03-23T07:56:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.912+0000 7fc728200640 1 -- 172.21.15.105:0/631974093 wait complete. 2024-03-23T07:56:48.926 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-03-23T07:56:48.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-23T07:56:48.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.937+0000 7fb4e1400640 1 -- 172.21.15.105:0/535434322 --> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4b4002cb0 con 0x7fb4bc0741f0 2024-03-23T07:56:48.944 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-03-23T07:56:48.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.939+0000 7fb4d2a00640 1 -- 172.21.15.105:0/535434322 <== osd.6 v2:172.21.15.132:6808/37442 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4b4002cb0 con 0x7fb4bc0741f0 2024-03-23T07:56:48.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.942+0000 7fb4c7e00640 1 -- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7fb4bc0741f0 msgr2=0x7fb4bc076630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.942+0000 7fb4c7e00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7fb4bc0741f0 0x7fb4bc076630 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.942+0000 7fb4c7e00640 1 -- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb4bc06ac30 msgr2=0x7fb4bc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.942+0000 7fb4c7e00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb4bc06ac30 0x7fb4bc06d0f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc035000 tx=0x7fb4cc008030 comp rx=0 tx=0).stop 2024-03-23T07:56:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.942+0000 7fb4c7e00640 1 -- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4dc15cc00 msgr2=0x7fb4dc0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.942+0000 7fb4c7e00640 1 --2- 172.21.15.105:0/535434322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4dc15cc00 0x7fb4dc0cc730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400ff70 tx=0x7fb4d4010550 comp rx=0 tx=0).stop 2024-03-23T07:56:48.945 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-23T07:56:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.943+0000 7fb4db200640 1 -- 172.21.15.105:0/535434322 reap_dead start 2024-03-23T07:56:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.944+0000 7fb4c7e00640 1 -- 172.21.15.105:0/535434322 shutdown_connections 2024-03-23T07:56:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.944+0000 7fb4c7e00640 1 -- 172.21.15.105:0/535434322 >> 172.21.15.105:0/535434322 conn(0x7fb4dc069c90 msgr2=0x7fb4dc153570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.944+0000 7fb4c7e00640 1 -- 172.21.15.105:0/535434322 shutdown_connections 2024-03-23T07:56:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:48.944+0000 7fb4c7e00640 1 -- 172.21.15.105:0/535434322 wait complete. 2024-03-23T07:56:48.967 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-03-23T07:56:48.968 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-23T07:56:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.181+0000 7f2e3b200640 1 Processor -- start 2024-03-23T07:56:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.181+0000 7f2e3b200640 1 -- start start 2024-03-23T07:56:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.181+0000 7f5039400640 1 Processor -- start 2024-03-23T07:56:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.183+0000 7f2e3b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c15c820 0x7f2e3c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.183+0000 7f2e3b200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e3c160e90 0x7f2e3c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.183+0000 7f2e3b200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c0cfa30 con 0x7f2e3c0cf5f0 2024-03-23T07:56:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.183+0000 7f2e3b200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c0cdd70 con 0x7f2e3c160e90 2024-03-23T07:56:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.183+0000 7f2e3b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0ccd90 con 0x7f2e3c15c820 2024-03-23T07:56:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.184+0000 7f2e39400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e3c160e90 0x7f2e3c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.184+0000 7f2e39400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e3c160e90 0x7f2e3c0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33414/0 (socket says 172.21.15.105:33414) 2024-03-23T07:56:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.184+0000 7f2e39400640 1 -- 172.21.15.105:0/856104059 learned_addr learned my addr 172.21.15.105:0/856104059 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.182+0000 7f5039400640 1 -- start start 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.184+0000 7f5039400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50340d04b0 0x7f50340c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.184+0000 7f5039400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50340b3a80 con 0x7f50340ce360 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.184+0000 7f5039400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50340b3070 con 0x7f50340cefe0 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.184+0000 7f5039400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50340b3e20 con 0x7f50340d04b0 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.185+0000 7f5033400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50340d04b0 0x7f50340c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.185+0000 7f5038a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f50340ce360 0x7f50340b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59462/0 (socket says 172.21.15.105:59462) 2024-03-23T07:56:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.185+0000 7f5038a00640 1 -- 172.21.15.105:0/623308100 learned_addr learned my addr 172.21.15.105:0/623308100 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.185+0000 7f2e39e00640 1 --2- 172.21.15.105:0/856104059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c15c820 0x7f2e3c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.187+0000 7f5033400640 1 -- 172.21.15.105:0/623308100 >> v1:172.21.15.132:6789/0 conn(0x7f50340cefe0 legacy=0x7f50340d0d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T07:56:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.187+0000 7f5033400640 1 -- 172.21.15.105:0/623308100 >> v1:172.21.15.105:6789/0 conn(0x7f50340ce360 legacy=0x7f50340b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.187+0000 7f5033400640 1 -- 172.21.15.105:0/623308100 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50340cfa40 con 0x7f50340d04b0 2024-03-23T07:56:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.187+0000 7f2e39400640 1 -- 172.21.15.105:0/856104059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c15c820 msgr2=0x7f2e3c0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.187+0000 7f2e39400640 1 --2- 172.21.15.105:0/856104059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c15c820 0x7f2e3c0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.187+0000 7f2e39400640 1 -- 172.21.15.105:0/856104059 >> v1:172.21.15.105:6789/0 conn(0x7f2e3c0cf5f0 legacy=0x7f2e3c0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.187+0000 7f2e39400640 1 -- 172.21.15.105:0/856104059 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c0b7fc0 con 0x7f2e3c160e90 2024-03-23T07:56:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.189+0000 7f2e39400640 1 --2- 172.21.15.105:0/856104059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c160e90 0x7f2e3c0cf0b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009870 tx=0x7f2e2c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b656c4e91de8b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.191+0000 7f2e38a00640 1 -- 172.21.15.105:0/856104059 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c02f850 con 0x7f2e3c160e90 2024-03-23T07:56:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.191+0000 7f5033400640 1 --2- 172.21.15.105:0/623308100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340d04b0 0x7f50340c5990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d90 tx=0x7f502c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf352cb97b6be7b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.191+0000 7f2e38a00640 1 -- 172.21.15.105:0/856104059 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c02f9b0 con 0x7f2e3c160e90 2024-03-23T07:56:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.192+0000 7f2e38a00640 1 -- 172.21.15.105:0/856104059 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c02fd80 con 0x7f2e3c160e90 2024-03-23T07:56:49.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.193+0000 7f2e3b200640 1 -- 172.21.15.105:0/856104059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c160e90 msgr2=0x7f2e3c0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.193+0000 7f5032a00640 1 -- 172.21.15.105:0/623308100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c02cbd0 con 0x7f50340d04b0 2024-03-23T07:56:49.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.193+0000 7f2e3b200640 1 --2- 172.21.15.105:0/856104059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c160e90 0x7f2e3c0cf0b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009870 tx=0x7f2e2c02ba30 comp rx=0 tx=0).stop 2024-03-23T07:56:49.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.194+0000 7f5032a00640 1 -- 172.21.15.105:0/623308100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c02cd30 con 0x7f50340d04b0 2024-03-23T07:56:49.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.194+0000 7f5032a00640 1 -- 172.21.15.105:0/623308100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c02cea0 con 0x7f50340d04b0 2024-03-23T07:56:49.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f2e3b200640 1 -- 172.21.15.105:0/856104059 shutdown_connections 2024-03-23T07:56:49.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f2e3b200640 1 --2- 172.21.15.105:0/856104059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c160e90 0x7f2e3c0cf0b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f2e3b200640 1 --2- 172.21.15.105:0/856104059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c15c820 0x7f2e3c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f2e3b200640 1 -- 172.21.15.105:0/856104059 >> 172.21.15.105:0/856104059 conn(0x7f2e3c06b070 msgr2=0x7f2e3c06b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f2e3b200640 1 -- 172.21.15.105:0/856104059 shutdown_connections 2024-03-23T07:56:49.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f5039400640 1 -- 172.21.15.105:0/623308100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340d04b0 msgr2=0x7f50340c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f5039400640 1 --2- 172.21.15.105:0/623308100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340d04b0 0x7f50340c5990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d90 tx=0x7f502c02be20 comp rx=0 tx=0).stop 2024-03-23T07:56:49.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f5039400640 1 -- 172.21.15.105:0/623308100 shutdown_connections 2024-03-23T07:56:49.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.196+0000 7f2e3b200640 1 -- 172.21.15.105:0/856104059 wait complete. 2024-03-23T07:56:49.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f5039400640 1 --2- 172.21.15.105:0/623308100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340d04b0 0x7f50340c5990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.195+0000 7f5039400640 1 -- 172.21.15.105:0/623308100 >> 172.21.15.105:0/623308100 conn(0x7f5034089fd0 msgr2=0x7f50340d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.196+0000 7f5039400640 1 -- 172.21.15.105:0/623308100 shutdown_connections 2024-03-23T07:56:49.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.196+0000 7f5039400640 1 -- 172.21.15.105:0/623308100 wait complete. 2024-03-23T07:56:49.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.199+0000 7f5039400640 1 Processor -- start 2024-03-23T07:56:49.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.199+0000 7f5039400640 1 -- start start 2024-03-23T07:56:49.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.200+0000 7f2e3b200640 1 Processor -- start 2024-03-23T07:56:49.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.200+0000 7f2e3b200640 1 -- start start 2024-03-23T07:56:49.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.201+0000 7f5039400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340ce360 0x7f50341d8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.201+0000 7f5039400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50340cefe0 0x7f50340cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.202+0000 7f5039400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50340cb710 0x7f50341eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.202+0000 7f5039400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50340aff80 con 0x7f50340ce360 2024-03-23T07:56:49.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.202+0000 7f5038a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50340cb710 0x7f50341eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.203+0000 7f5038a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50340cb710 0x7f50341eb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38052/0 (socket says 172.21.15.105:38052) 2024-03-23T07:56:49.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.203+0000 7f5038a00640 1 -- 172.21.15.105:0/2515177457 learned_addr learned my addr 172.21.15.105:0/2515177457 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.203+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50340cbca0 con 0x7f50340cb710 2024-03-23T07:56:49.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.203+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50340cbf80 con 0x7f50340cefe0 2024-03-23T07:56:49.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.203+0000 7f5033400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50340cefe0 0x7f50340cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.203+0000 7f5033e00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340ce360 0x7f50341d8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f2e3b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c0cf5f0 0x7f2e3c14fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f5038a00640 1 -- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50340cefe0 msgr2=0x7f50340cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f5038a00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50340cefe0 0x7f50340cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.205+0000 7f5038a00640 1 -- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340ce360 msgr2=0x7f50341d8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.205+0000 7f5038a00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340ce360 0x7f50341d8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.205+0000 7f5038a00640 1 -- 172.21.15.105:0/2515177457 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c009a70 con 0x7f50340cb710 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.205+0000 7f5033400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50340cefe0 0x7f50340cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f2e3b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15c820 0x7f2e3c152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f2e3b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e3c160e90 0x7f2e3c14b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f2e3b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0b2660 con 0x7f2e3c15c820 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f2e3b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c152d20 con 0x7f2e3c160e90 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.204+0000 7f2e3b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0b1d70 con 0x7f2e3c0cf5f0 2024-03-23T07:56:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.205+0000 7f5033e00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340ce360 0x7f50341d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.205+0000 7f2e39e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c0cf5f0 0x7f2e3c14fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.206+0000 7f2e39400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15c820 0x7f2e3c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.205+0000 7f2e3a800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e3c160e90 0x7f2e3c14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.206+0000 7f2e39400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15c820 0x7f2e3c152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33452/0 (socket says 172.21.15.105:33452) 2024-03-23T07:56:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.206+0000 7f2e39400640 1 -- 172.21.15.105:0/2411843941 learned_addr learned my addr 172.21.15.105:0/2411843941 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.206+0000 7f5038a00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50340cb710 0x7f50341eb1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f502400c390 tx=0x7f502400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.207+0000 7f502be00640 1 -- 172.21.15.105:0/2515177457 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024005ce0 con 0x7f50340cb710 2024-03-23T07:56:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.207+0000 7f502be00640 1 -- 172.21.15.105:0/2515177457 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5024005e40 con 0x7f50340cb710 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.207+0000 7f2e3a800640 1 -- 172.21.15.105:0/2411843941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c0cf5f0 msgr2=0x7f2e3c14fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.207+0000 7f2e3a800640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c0cf5f0 0x7f2e3c14fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.207+0000 7f2e3a800640 1 -- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15c820 msgr2=0x7f2e3c152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.207+0000 7f2e3a800640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15c820 0x7f2e3c152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.207+0000 7f2e3a800640 1 -- 172.21.15.105:0/2411843941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0094d0 con 0x7f2e3c160e90 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.208+0000 7f2e3a800640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e3c160e90 0x7f2e3c14b330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c3c0 tx=0x7f2e2800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.208+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50340b0190 con 0x7f50340cb710 2024-03-23T07:56:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.208+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50341e5b00 con 0x7f50340cb710 2024-03-23T07:56:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.208+0000 7f502be00640 1 -- 172.21.15.105:0/2515177457 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50240072d0 con 0x7f50340cb710 2024-03-23T07:56:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.209+0000 7f2e32000640 1 -- 172.21.15.105:0/2411843941 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e280052d0 con 0x7f2e3c160e90 2024-03-23T07:56:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.210+0000 7f2e32000640 1 -- 172.21.15.105:0/2411843941 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e28005430 con 0x7f2e3c160e90 2024-03-23T07:56:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.210+0000 7f2e32000640 1 -- 172.21.15.105:0/2411843941 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28007240 con 0x7f2e3c160e90 2024-03-23T07:56:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.210+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c152a80 con 0x7f2e3c160e90 2024-03-23T07:56:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.210+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c0b22f0 con 0x7f2e3c160e90 2024-03-23T07:56:49.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.213+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c005250 con 0x7f50340cb710 2024-03-23T07:56:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.214+0000 7f502be00640 1 -- 172.21.15.105:0/2515177457 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f50240074f0 con 0x7f50340cb710 2024-03-23T07:56:49.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.217+0000 7f502be00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f501406acd0 0x7f501406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.217+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e14005250 con 0x7f2e3c160e90 2024-03-23T07:56:49.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.217+0000 7f2e32000640 1 -- 172.21.15.105:0/2411843941 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2e280073a0 con 0x7f2e3c160e90 2024-03-23T07:56:49.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.218+0000 7f502be00640 1 -- 172.21.15.105:0/2515177457 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5024088a30 con 0x7f50340cb710 2024-03-23T07:56:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.218+0000 7f5033e00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f501406acd0 0x7f501406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.219+0000 7f9e7f400640 1 Processor -- start 2024-03-23T07:56:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.219+0000 7f2e32000640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2e1c06ad00 0x7f2e1c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.220+0000 7f2e32000640 1 -- 172.21.15.105:0/2411843941 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2e28088690 con 0x7f2e3c160e90 2024-03-23T07:56:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.220+0000 7f5033e00640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f501406acd0 0x7f501406d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5020006fd0 tx=0x7f50200090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.221+0000 7f2e39e00640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2e1c06ad00 0x7f2e1c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.223+0000 7f2e39e00640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2e1c06ad00 0x7f2e1c06d1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0b1410 tx=0x7f2e3400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.219+0000 7f9e7f400640 1 -- start start 2024-03-23T07:56:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.221+0000 7f9e7f400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9e7815cc70 0x7f9e780cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.221+0000 7f9e7f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e780b4e60 0x7f9e780d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.221+0000 7f9e7f400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e781607d0 con 0x7f9e780b1f20 2024-03-23T07:56:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.221+0000 7f9e7f400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e78160a70 con 0x7f9e7815cc70 2024-03-23T07:56:49.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.221+0000 7f9e7f400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78160d50 con 0x7f9e780b4e60 2024-03-23T07:56:49.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.225+0000 7f9e7ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9e7815cc70 0x7f9e780cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.225+0000 7f9e7ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9e7815cc70 0x7f9e780cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38068/0 (socket says 172.21.15.105:38068) 2024-03-23T07:56:49.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.225+0000 7f9e7ca00640 1 -- 172.21.15.105:0/3608499861 learned_addr learned my addr 172.21.15.105:0/3608499861 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.226+0000 7f9e7d400640 1 --2- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e780b4e60 0x7f9e780d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.226+0000 7f9e7ca00640 1 -- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e780b4e60 msgr2=0x7f9e780d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:49.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.226+0000 7f9e7ca00640 1 --2- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e780b4e60 0x7f9e780d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.226+0000 7f9e7ca00640 1 -- 172.21.15.105:0/3608499861 >> v1:172.21.15.132:6789/0 conn(0x7f9e780b1f20 legacy=0x7f9e780b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.226+0000 7f9e7ca00640 1 -- 172.21.15.105:0/3608499861 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e78161030 con 0x7f9e7815cc70 2024-03-23T07:56:49.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.228+0000 7f9e7ca00640 1 --2- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e7815cc70 0x7f9e780cdba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009870 tx=0x7f9e6c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91aeca668e1e69f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.231+0000 7f5db2e00640 1 Processor -- start 2024-03-23T07:56:49.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.231+0000 7f5db2e00640 1 -- start start 2024-03-23T07:56:49.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.232+0000 7f5db2e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5dac15c820 0x7f5dac0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.232+0000 7f5db2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.232+0000 7f5db2e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dac0ccd90 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.232+0000 7f5db2e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dac0cdd70 con 0x7f5dac15c820 2024-03-23T07:56:49.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.232+0000 7f5db2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0cfa60 con 0x7f5dac160e90 2024-03-23T07:56:49.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.232+0000 7f5db2400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5dac0cf5f0 0x7f5dac0d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59478/0 (socket says 172.21.15.105:59478) 2024-03-23T07:56:49.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.232+0000 7f5db2400640 1 -- 172.21.15.105:0/3393458364 learned_addr learned my addr 172.21.15.105:0/3393458364 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:49.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5db1a00640 1 --2- 172.21.15.105:0/3393458364 >> v2:172.21.15.132:3300/0 conn(0x7f5dac15c820 0x7f5dac0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5db1000640 1 --2- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5dabe00640 1 -- 172.21.15.105:0/3393458364 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388744476 0 0) 0x7f5dac0ccd90 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5dabe00640 1 -- 172.21.15.105:0/3393458364 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d94003520 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5db1a00640 1 -- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 msgr2=0x7f5dac0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5db1a00640 1 --2- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac0cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5db1a00640 1 -- 172.21.15.105:0/3393458364 >> v1:172.21.15.105:6789/0 conn(0x7f5dac0cf5f0 legacy=0x7f5dac0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.233+0000 7f5db1a00640 1 -- 172.21.15.105:0/3393458364 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0b7fc0 con 0x7f5dac15c820 2024-03-23T07:56:49.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.234+0000 7f502be00640 1 -- 172.21.15.105:0/2515177457 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5024053a40 con 0x7f50340cb710 2024-03-23T07:56:49.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.236+0000 7f5db1a00640 1 --2- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac0cd830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5da00097a0 tx=0x7f5da002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe122286b8ece6c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.237+0000 7f51f3200640 1 Processor -- start 2024-03-23T07:56:49.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.237+0000 7f51f3200640 1 -- start start 2024-03-23T07:56:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.238+0000 7f51f3200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51e403a200 0x7f51e403a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.239+0000 7f51f3200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e4055c60 con 0x7f51e400a1d0 2024-03-23T07:56:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.239+0000 7f51f3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e4055370 con 0x7f51e4009df0 2024-03-23T07:56:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.239+0000 7f51f3200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e4055610 con 0x7f51e403a200 2024-03-23T07:56:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.239+0000 7f51eb400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f51e400a1d0 0x7f51e4051cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59482/0 (socket says 172.21.15.105:59482) 2024-03-23T07:56:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.239+0000 7f51eb400640 1 -- 172.21.15.105:0/1330315100 learned_addr learned my addr 172.21.15.105:0/1330315100 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.239+0000 7f51f1200640 1 --2- 172.21.15.105:0/1330315100 >> v2:172.21.15.105:3300/0 conn(0x7f51e403a200 0x7f51e403a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.240+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048128132 0 0) 0x7f51e4055c60 con 0x7f51e400a1d0 2024-03-23T07:56:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.240+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d4003520 con 0x7f51e400a1d0 2024-03-23T07:56:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.240+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784641480 0 0) 0x7f51d4003520 con 0x7f51e400a1d0 2024-03-23T07:56:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.240+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 >> v2:172.21.15.105:3300/0 conn(0x7f51e403a200 msgr2=0x7f51e403a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.240+0000 7f51eaa00640 1 --2- 172.21.15.105:0/1330315100 >> v2:172.21.15.105:3300/0 conn(0x7f51e403a200 0x7f51e403a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.240+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 >> v1:172.21.15.132:6789/0 conn(0x7f51e4009df0 legacy=0x7f51e40560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40558f0 con 0x7f51e400a1d0 2024-03-23T07:56:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f51d8001670 con 0x7f51e400a1d0 2024-03-23T07:56:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 >> v1:172.21.15.105:6789/0 conn(0x7f51e400a1d0 legacy=0x7f51e4051cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d40047c0 0x7f51d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51d40052a0 0x7f51d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4055610 con 0x7f51d40047c0 2024-03-23T07:56:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d400c1a0 con 0x7f51d40052a0 2024-03-23T07:56:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d400c360 con 0x7f51d40039c0 2024-03-23T07:56:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51eb400640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d40047c0 0x7f51d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51ebe00640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.241+0000 7f51f1200640 1 --2- 172.21.15.105:0/1330315100 >> v2:172.21.15.105:3300/0 conn(0x7f51e403a200 0x7f51e403a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.242+0000 7f51ebe00640 1 -- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51d40052a0 msgr2=0x7f51d400b980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T07:56:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.242+0000 7f51ebe00640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51d40052a0 0x7f51d400b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.242+0000 7f51ebe00640 1 -- 172.21.15.105:0/1330315100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d40047c0 msgr2=0x7f51d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.242+0000 7f51ebe00640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d40047c0 0x7f51d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.242+0000 7f51ebe00640 1 -- 172.21.15.105:0/1330315100 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40558f0 con 0x7f51d40039c0 2024-03-23T07:56:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.243+0000 7f51f1200640 1 -- 172.21.15.105:0/1330315100 reap_dead start 2024-03-23T07:56:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.243+0000 7f51ebe00640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51d4003de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51e00016a0 tx=0x7f51d40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95f1893cf10e47fb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.243+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00052d0 con 0x7f51d40039c0 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.244+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e002e050 con 0x7f51d40039c0 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.244+0000 7f51eaa00640 1 -- 172.21.15.105:0/1330315100 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0003640 con 0x7f51d40039c0 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.244+0000 7f51f3200640 1 -- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 msgr2=0x7f51d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.244+0000 7f51f3200640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51d4003de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51e00016a0 tx=0x7f51d40057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.244+0000 7f51f3200640 1 -- 172.21.15.105:0/1330315100 shutdown_connections 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.244+0000 7f51f3200640 1 --2- 172.21.15.105:0/1330315100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51d4003de0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.244+0000 7f51f3200640 1 -- 172.21.15.105:0/1330315100 >> 172.21.15.105:0/1330315100 conn(0x7f51e4053770 msgr2=0x7f51e4053b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.245+0000 7f51f3200640 1 -- 172.21.15.105:0/1330315100 shutdown_connections 2024-03-23T07:56:49.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.245+0000 7f51f3200640 1 -- 172.21.15.105:0/1330315100 wait complete. 2024-03-23T07:56:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.247+0000 7f51f3200640 1 Processor -- start 2024-03-23T07:56:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.247+0000 7f51f3200640 1 -- start start 2024-03-23T07:56:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.247+0000 7f9e77400640 1 -- 172.21.15.105:0/3608499861 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c030840 con 0x7f9e7815cc70 2024-03-23T07:56:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51f3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51e4048e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51f3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51e40e2e20 0x7f51e404dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51f3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51e40e3280 0x7f51e404e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51f3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40d1590 con 0x7f51e40e2e20 2024-03-23T07:56:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51f3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e404e940 con 0x7f51e40e3280 2024-03-23T07:56:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51f3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e404ec20 con 0x7f51d40039c0 2024-03-23T07:56:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51e4048e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51e4048e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52332/0 (socket says 172.21.15.105:52332) 2024-03-23T07:56:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f51ebe00640 1 -- 172.21.15.105:0/1553028133 learned_addr learned my addr 172.21.15.105:0/1553028133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f51f1200640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51e40e3280 0x7f51e404e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f51ebe00640 1 -- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51e40e3280 msgr2=0x7f51e404e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f51ebe00640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51e40e3280 0x7f51e404e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f51ebe00640 1 -- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51e40e2e20 msgr2=0x7f51e404dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f51ebe00640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51e40e2e20 0x7f51e404dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f51ebe00640 1 -- 172.21.15.105:0/1553028133 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e404ef00 con 0x7f51d40039c0 2024-03-23T07:56:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f2e32000640 1 -- 172.21.15.105:0/2411843941 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2e280536a0 con 0x7f2e3c160e90 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f9e77400640 1 -- 172.21.15.105:0/3608499861 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c0309a0 con 0x7f9e7815cc70 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.248+0000 7f9e77400640 1 -- 172.21.15.105:0/3608499861 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c030d30 con 0x7f9e7815cc70 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f9e7f400640 1 -- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e7815cc70 msgr2=0x7f9e780cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e7815cc70 0x7f9e780cdba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009870 tx=0x7f9e6c02b980 comp rx=0 tx=0).stop 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f9e7f400640 1 -- 172.21.15.105:0/3608499861 shutdown_connections 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e780b4e60 0x7f9e780d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3608499861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e7815cc70 0x7f9e780cdba0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.249+0000 7f9e7f400640 1 -- 172.21.15.105:0/3608499861 >> 172.21.15.105:0/3608499861 conn(0x7f9e780cf790 msgr2=0x7f9e780cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f51ebe00640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51e4048e80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51e002bb40 tx=0x7f51e00010a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f5dabe00640 1 -- 172.21.15.105:0/3393458364 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0030800 con 0x7f5dac15c820 2024-03-23T07:56:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f031f400640 1 Processor -- start 2024-03-23T07:56:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f5dabe00640 1 -- 172.21.15.105:0/3393458364 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da0030960 con 0x7f5dac15c820 2024-03-23T07:56:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f5dabe00640 1 -- 172.21.15.105:0/3393458364 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0030cf0 con 0x7f5dac15c820 2024-03-23T07:56:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.251+0000 7f51dfe00640 1 -- 172.21.15.105:0/1553028133 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0006ea0 con 0x7f51d40039c0 2024-03-23T07:56:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.251+0000 7f51dfe00640 1 -- 172.21.15.105:0/1553028133 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0007250 con 0x7f51d40039c0 2024-03-23T07:56:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.251+0000 7f51dfe00640 1 -- 172.21.15.105:0/1553028133 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00075c0 con 0x7f51d40039c0 2024-03-23T07:56:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.251+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e404f6e0 con 0x7f51d40039c0 2024-03-23T07:56:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.251+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e404b9d0 con 0x7f51d40039c0 2024-03-23T07:56:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f5db2e00640 1 -- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 msgr2=0x7f5dac0cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f5db2e00640 1 --2- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac0cd830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5da00097a0 tx=0x7f5da002b940 comp rx=0 tx=0).stop 2024-03-23T07:56:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f9e7f400640 1 -- 172.21.15.105:0/3608499861 shutdown_connections 2024-03-23T07:56:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f9e7f400640 1 -- 172.21.15.105:0/3608499861 wait complete. 2024-03-23T07:56:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f5db2e00640 1 -- 172.21.15.105:0/3393458364 shutdown_connections 2024-03-23T07:56:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f5db2e00640 1 --2- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f5db2e00640 1 --2- 172.21.15.105:0/3393458364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac0cd830 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f5db2e00640 1 -- 172.21.15.105:0/3393458364 >> 172.21.15.105:0/3393458364 conn(0x7f5dac06b070 msgr2=0x7f5dac06b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f5db2e00640 1 -- 172.21.15.105:0/3393458364 shutdown_connections 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.250+0000 7f031f400640 1 -- start start 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f031f400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03180cf300 0x7f03180ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f031f400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03180cea20 con 0x7f031815a960 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f031f400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0318099dd0 con 0x7f03180cbc70 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f031f400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03180cc5e0 con 0x7f03180cf300 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.253+0000 7f031ca00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f031815a960 0x7f03180996c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59486/0 (socket says 172.21.15.105:59486) 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.253+0000 7f031ca00640 1 -- 172.21.15.105:0/3792837715 learned_addr learned my addr 172.21.15.105:0/3792837715 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.253+0000 7f031d400640 1 --2- 172.21.15.105:0/3792837715 >> v2:172.21.15.105:3300/0 conn(0x7f03180cf300 0x7f03180ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.254+0000 7f031d400640 1 -- 172.21.15.105:0/3792837715 >> v1:172.21.15.132:6789/0 conn(0x7f03180cbc70 legacy=0x7f03180cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.254+0000 7f031d400640 1 -- 172.21.15.105:0/3792837715 >> v1:172.21.15.105:6789/0 conn(0x7f031815a960 legacy=0x7f03180996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.254+0000 7f0317400640 1 -- 172.21.15.105:0/3792837715 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435432766 0 0) 0x7f0304001110 con 0x7f031815a960 2024-03-23T07:56:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.252+0000 7f9e7f400640 1 Processor -- start 2024-03-23T07:56:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.255+0000 7f9e7f400640 1 -- start start 2024-03-23T07:56:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.253+0000 7f5db2e00640 1 -- 172.21.15.105:0/3393458364 wait complete. 2024-03-23T07:56:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.255+0000 7f51dfe00640 1 -- 172.21.15.105:0/1553028133 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f51e0003640 con 0x7f51d40039c0 2024-03-23T07:56:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.255+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e404fc00 con 0x7f51d40039c0 2024-03-23T07:56:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.255+0000 7f51dfe00640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f51cc06ac80 0x7f51cc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.256+0000 7f51dfe00640 1 -- 172.21.15.105:0/1553028133 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f51e007a530 con 0x7f51d40039c0 2024-03-23T07:56:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.256+0000 7f51eb400640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f51cc06ac80 0x7f51cc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.256+0000 7f9e7f400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e780b1f20 0x7f9e780b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.256+0000 7f9e7f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e780b4e60 0x7f9e780cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.256+0000 7f9e7f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e7815cc70 0x7f9e780cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.257+0000 7f51eb400640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f51cc06ac80 0x7f51cc06d140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51d8001120 tx=0x7f51d8001a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.257+0000 7f9e7f400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78154270 con 0x7f9e780b4e60 2024-03-23T07:56:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.258+0000 7f031d400640 1 -- 172.21.15.105:0/3792837715 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180cc300 con 0x7f03180cf300 2024-03-23T07:56:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.257+0000 7f9e7ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e780b1f20 0x7f9e780b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.258+0000 7f9e7d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e7815cc70 0x7f9e780cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.258+0000 7f9e7d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e7815cc70 0x7f9e780cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52334/0 (socket says 172.21.15.105:52334) 2024-03-23T07:56:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.258+0000 7f9e7d400640 1 -- 172.21.15.105:0/3766599968 learned_addr learned my addr 172.21.15.105:0/3766599968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.258+0000 7f9e77e00640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e780b4e60 0x7f9e780cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.258+0000 7f5db2e00640 1 Processor -- start 2024-03-23T07:56:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.258+0000 7f9e7ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e780b1f20 0x7f9e780b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38112/0 (socket says 172.21.15.105:38112) 2024-03-23T07:56:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.259+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78071220 con 0x7f9e780b1f20 2024-03-23T07:56:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.259+0000 7f5db2e00640 1 -- start start 2024-03-23T07:56:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.259+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78070610 con 0x7f9e7815cc70 2024-03-23T07:56:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f031d400640 1 --2- 172.21.15.105:0/3792837715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03180cf300 0x7f03180ce4e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0310009e10 tx=0x7f0310030380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b07a24bf870334 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f0317400640 1 -- 172.21.15.105:0/3792837715 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310007740 con 0x7f03180cf300 2024-03-23T07:56:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f0317400640 1 -- 172.21.15.105:0/3792837715 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0310033050 con 0x7f03180cf300 2024-03-23T07:56:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f0317400640 1 -- 172.21.15.105:0/3792837715 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310007740 con 0x7f03180cf300 2024-03-23T07:56:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f5db2e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dac0cf5f0 0x7f5dac1f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f9e7ca00640 1 -- 172.21.15.105:0/3766599968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e7815cc70 msgr2=0x7f9e780cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f9e7ca00640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e7815cc70 0x7f9e780cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f9e7ca00640 1 -- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e780b4e60 msgr2=0x7f9e780cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f9e7ca00640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e780b4e60 0x7f9e780cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.260+0000 7f9e7ca00640 1 -- 172.21.15.105:0/3766599968 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0094d0 con 0x7f9e780b1f20 2024-03-23T07:56:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f031f400640 1 -- 172.21.15.105:0/3792837715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03180cf300 msgr2=0x7f03180ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f031f400640 1 --2- 172.21.15.105:0/3792837715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03180cf300 0x7f03180ce4e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0310009e10 tx=0x7f0310030380 comp rx=0 tx=0).stop 2024-03-23T07:56:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f5db2e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f5db2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f5db2e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac1f2040 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f5db2e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac15d580 con 0x7f5dac15c820 2024-03-23T07:56:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f5db2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac15fc90 con 0x7f5dac160e90 2024-03-23T07:56:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f031f400640 1 -- 172.21.15.105:0/3792837715 shutdown_connections 2024-03-23T07:56:49.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f031f400640 1 --2- 172.21.15.105:0/3792837715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03180cf300 0x7f03180ce4e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f031f400640 1 -- 172.21.15.105:0/3792837715 >> 172.21.15.105:0/3792837715 conn(0x7f03180b2140 msgr2=0x7f03180b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.261+0000 7f5db1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dac0cf5f0 0x7f5dac1f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f5db1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dac0cf5f0 0x7f5dac1f1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33510/0 (socket says 172.21.15.105:33510) 2024-03-23T07:56:49.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f5db1a00640 1 -- 172.21.15.105:0/2521128202 learned_addr learned my addr 172.21.15.105:0/2521128202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f9e7ca00640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e780b1f20 0x7f9e780b6e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c033000 tx=0x7f9e6c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f031f400640 1 -- 172.21.15.105:0/3792837715 shutdown_connections 2024-03-23T07:56:49.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f031f400640 1 -- 172.21.15.105:0/3792837715 wait complete. 2024-03-23T07:56:49.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f5db1000640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f5db2400640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.262+0000 7f9e74c00640 1 -- 172.21.15.105:0/3766599968 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c03e040 con 0x7f9e780b1f20 2024-03-23T07:56:49.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f9e74c00640 1 -- 172.21.15.105:0/3766599968 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c0443f0 con 0x7f9e780b1f20 2024-03-23T07:56:49.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f9e74c00640 1 -- 172.21.15.105:0/3766599968 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0447a0 con 0x7f9e780b1f20 2024-03-23T07:56:49.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f5db1a00640 1 -- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 msgr2=0x7f5dac15f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f5db1a00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac15f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f5db1a00640 1 -- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 msgr2=0x7f5dac15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f5db1a00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac15f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f5db1a00640 1 -- 172.21.15.105:0/2521128202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da0009400 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f5db1000640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:49.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f5db1a00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dac0cf5f0 0x7f5dac1f1b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5da002bf50 tx=0x7f5da0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e780708f0 con 0x7f9e780b1f20 2024-03-23T07:56:49.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.263+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e78070eb0 con 0x7f9e780b1f20 2024-03-23T07:56:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.264+0000 7f5da9600640 1 -- 172.21.15.105:0/2521128202 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da003e040 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.264+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dac1541c0 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.264+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac15d7d0 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.264+0000 7f5da9600640 1 -- 172.21.15.105:0/2521128202 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da0039c10 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.264+0000 7f5da9600640 1 -- 172.21.15.105:0/2521128202 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da003e040 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.265+0000 7f031f400640 1 Processor -- start 2024-03-23T07:56:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.265+0000 7f031f400640 1 -- start start 2024-03-23T07:56:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.266+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88005250 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.268+0000 7f5da9600640 1 -- 172.21.15.105:0/2521128202 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5da004d3b0 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.269+0000 7f5da9600640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5d8c06ac60 0x7f5d8c06d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.269+0000 7f5da9600640 1 -- 172.21.15.105:0/2521128202 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5da00ab8f0 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.270+0000 7f9e74c00640 1 -- 172.21.15.105:0/3766599968 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e6c043390 con 0x7f9e780b1f20 2024-03-23T07:56:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.270+0000 7f5db1000640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5d8c06ac60 0x7f5d8c06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.270+0000 7f031f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03180cbc70 0x7f0318158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.271+0000 7f031f400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03180cf300 0x7f031815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.271+0000 7f031f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f031815a960 0x7f031815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.271+0000 7f031f400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c9db0 con 0x7f031815a960 2024-03-23T07:56:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.271+0000 7f031f400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c9270 con 0x7f03180cf300 2024-03-23T07:56:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.272+0000 7f031ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03180cbc70 0x7f0318158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.272+0000 7f031ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03180cbc70 0x7f0318158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52358/0 (socket says 172.21.15.105:52358) 2024-03-23T07:56:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.272+0000 7f031ca00640 1 -- 172.21.15.105:0/782521722 learned_addr learned my addr 172.21.15.105:0/782521722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.272+0000 7f5db1000640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5d8c06ac60 0x7f5d8c06d120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0c9220 tx=0x7f5d98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.272+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0318158980 con 0x7f03180cbc70 2024-03-23T07:56:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e54005250 con 0x7f9e780b1f20 2024-03-23T07:56:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f031d400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f031815a960 0x7f031815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f031ca00640 1 -- 172.21.15.105:0/782521722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03180cf300 msgr2=0x7f031815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f031ca00640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03180cf300 0x7f031815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f031ca00640 1 -- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f031815a960 msgr2=0x7f031815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f031ca00640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f031815a960 0x7f031815cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f031ca00640 1 -- 172.21.15.105:0/782521722 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0310009a70 con 0x7f03180cbc70 2024-03-23T07:56:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.273+0000 7f031d400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f031815a960 0x7f031815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.274+0000 7f031ca00640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03180cbc70 0x7f0318158440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f030400c430 tx=0x7f030400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.274+0000 7f9e74c00640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9e5806ac00 0x7f9e5806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.274+0000 7f9e74c00640 1 -- 172.21.15.105:0/3766599968 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e6c0abbc0 con 0x7f9e780b1f20 2024-03-23T07:56:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.275+0000 7f0314c00640 1 -- 172.21.15.105:0/782521722 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040074e0 con 0x7f03180cbc70 2024-03-23T07:56:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.275+0000 7f0314c00640 1 -- 172.21.15.105:0/782521722 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0304007660 con 0x7f03180cbc70 2024-03-23T07:56:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.275+0000 7f0314c00640 1 -- 172.21.15.105:0/782521722 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304004a20 con 0x7f03180cbc70 2024-03-23T07:56:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.275+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03180c9510 con 0x7f03180cbc70 2024-03-23T07:56:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.276+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03180c9ad0 con 0x7f03180cbc70 2024-03-23T07:56:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.276+0000 7f9e77e00640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9e5806ac00 0x7f9e5806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.276+0000 7f51dfe00640 1 -- 172.21.15.105:0/1553028133 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f51e0076900 con 0x7f51d40039c0 2024-03-23T07:56:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.277+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03180c4210 con 0x7f03180cbc70 2024-03-23T07:56:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.278+0000 7f9e77e00640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9e5806ac00 0x7f9e5806d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9e780b83a0 tx=0x7f9e64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.281+0000 7f5da9600640 1 -- 172.21.15.105:0/2521128202 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5da00767d0 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.279+0000 7f0314c00640 1 -- 172.21.15.105:0/782521722 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0304004b80 con 0x7f03180cbc70 2024-03-23T07:56:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.281+0000 7f0314c00640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f02f806ac00 0x7f02f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.281+0000 7f0314c00640 1 -- 172.21.15.105:0/782521722 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f03040892d0 con 0x7f03180cbc70 2024-03-23T07:56:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.283+0000 7f0317e00640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f02f806ac00 0x7f02f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.284+0000 7f0317e00640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f02f806ac00 0x7f02f806d0c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f030c000f60 tx=0x7f030c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.290+0000 7f9e74c00640 1 -- 172.21.15.105:0/3766599968 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e6c076ac0 con 0x7f9e780b1f20 2024-03-23T07:56:49.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.295+0000 7f0314c00640 1 -- 172.21.15.105:0/782521722 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f03180c4210 con 0x7f03180cbc70 2024-03-23T07:56:49.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.316+0000 7f954a000640 1 Processor -- start 2024-03-23T07:56:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.321+0000 7f954a000640 1 -- start start 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.323+0000 7f954a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f954406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.323+0000 7f954a000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95440cd230 0x7f954406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.323+0000 7f954a000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95440b1930 0x7f95440b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.323+0000 7f954a000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95440ce2c0 con 0x7f95440b1930 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.323+0000 7f954a000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9544158b90 con 0x7f95440cd230 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.323+0000 7f954a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440cd740 con 0x7f95440710e0 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.324+0000 7f9549600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95440b1930 0x7f95440b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.324+0000 7f9543e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95440cd230 0x7f954406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.324+0000 7f9549600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95440b1930 0x7f95440b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33532/0 (socket says 172.21.15.105:33532) 2024-03-23T07:56:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.324+0000 7f9543e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95440cd230 0x7f954406cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38142/0 (socket says 172.21.15.105:38142) 2024-03-23T07:56:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.324+0000 7f9549600640 1 -- 172.21.15.105:0/2182397523 learned_addr learned my addr 172.21.15.105:0/2182397523 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.325+0000 7f9548c00640 1 --2- 172.21.15.105:0/2182397523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f954406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.327+0000 7fd402c00640 1 Processor -- start 2024-03-23T07:56:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.325+0000 7f9549600640 1 -- 172.21.15.105:0/2182397523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 msgr2=0x7f954406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.325+0000 7f9549600640 1 --2- 172.21.15.105:0/2182397523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f954406c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.325+0000 7f9549600640 1 -- 172.21.15.105:0/2182397523 >> v2:172.21.15.132:3300/0 conn(0x7f95440cd230 msgr2=0x7f954406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.325+0000 7f9549600640 1 --2- 172.21.15.105:0/2182397523 >> v2:172.21.15.132:3300/0 conn(0x7f95440cd230 0x7f954406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.325+0000 7f9549600640 1 -- 172.21.15.105:0/2182397523 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440cda20 con 0x7f95440b1930 2024-03-23T07:56:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.327+0000 7f9549600640 1 --2- 172.21.15.105:0/2182397523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f95440b1d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9544067370 tx=0x7f953c030590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beb74926eb904ea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.327+0000 7f9543400640 1 -- 172.21.15.105:0/2182397523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00a2d0 con 0x7f95440b1930 2024-03-23T07:56:49.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.327+0000 7f9543400640 1 -- 172.21.15.105:0/2182397523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c033050 con 0x7f95440b1930 2024-03-23T07:56:49.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.327+0000 7f9543400640 1 -- 172.21.15.105:0/2182397523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00a2d0 con 0x7f95440b1930 2024-03-23T07:56:49.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.328+0000 7f954a000640 1 -- 172.21.15.105:0/2182397523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 msgr2=0x7f95440b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.328+0000 7f954a000640 1 --2- 172.21.15.105:0/2182397523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f95440b1d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9544067370 tx=0x7f953c030590 comp rx=0 tx=0).stop 2024-03-23T07:56:49.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.329+0000 7fd402c00640 1 -- start start 2024-03-23T07:56:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.331+0000 7f954a000640 1 -- 172.21.15.105:0/2182397523 shutdown_connections 2024-03-23T07:56:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.331+0000 7f954a000640 1 --2- 172.21.15.105:0/2182397523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f95440b1d60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.331+0000 7f954a000640 1 --2- 172.21.15.105:0/2182397523 >> v2:172.21.15.132:3300/0 conn(0x7f95440cd230 0x7f954406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.331+0000 7f954a000640 1 --2- 172.21.15.105:0/2182397523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f954406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.331+0000 7f954a000640 1 -- 172.21.15.105:0/2182397523 >> 172.21.15.105:0/2182397523 conn(0x7f9544069c90 msgr2=0x7f954406a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.331+0000 7f954a000640 1 -- 172.21.15.105:0/2182397523 shutdown_connections 2024-03-23T07:56:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.331+0000 7f954a000640 1 -- 172.21.15.105:0/2182397523 wait complete. 2024-03-23T07:56:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.335+0000 7f954a000640 1 Processor -- start 2024-03-23T07:56:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.335+0000 7f954a000640 1 -- start start 2024-03-23T07:56:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.337+0000 7fd402c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3f40e32a0 0x7fd3f4052c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.337+0000 7fd402c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3f404ff50 0x7fd3f4037b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.337+0000 7f954a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f95440a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.337+0000 7f954a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f9544147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.337+0000 7fd402c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f40e3680 con 0x7fd3f4053160 2024-03-23T07:56:49.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.337+0000 7fd402c00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f4053670 con 0x7fd3f40e32a0 2024-03-23T07:56:49.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.337+0000 7fd402c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f4053950 con 0x7fd3f404ff50 2024-03-23T07:56:49.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7f954a000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95440cd230 0x7f9544157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7f9543e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f9544147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7f9543e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f9544147f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33546/0 (socket says 172.21.15.105:33546) 2024-03-23T07:56:49.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7f9548c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f95440a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7f9548c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f95440a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52376/0 (socket says 172.21.15.105:52376) 2024-03-23T07:56:49.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7f9543e00640 1 -- 172.21.15.105:0/4274406238 learned_addr learned my addr 172.21.15.105:0/4274406238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7fd402200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3f404ff50 0x7fd3f4037b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7fd402200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3f404ff50 0x7fd3f4037b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38158/0 (socket says 172.21.15.105:38158) 2024-03-23T07:56:49.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95441edb80 con 0x7f95440b1930 2024-03-23T07:56:49.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9544148460 con 0x7f95440cd230 2024-03-23T07:56:49.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440a7aa0 con 0x7f95440710e0 2024-03-23T07:56:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.338+0000 7fd402200640 1 -- 172.21.15.105:0/3128694399 learned_addr learned my addr 172.21.15.105:0/3128694399 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7fd402200640 1 -- 172.21.15.105:0/3128694399 >> v2:172.21.15.105:3300/0 conn(0x7fd3f40e32a0 msgr2=0x7fd3f4052c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T07:56:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7fd402200640 1 --2- 172.21.15.105:0/3128694399 >> v2:172.21.15.105:3300/0 conn(0x7fd3f40e32a0 0x7fd3f4052c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7fd402200640 1 -- 172.21.15.105:0/3128694399 >> v1:172.21.15.105:6789/0 conn(0x7fd3f4053160 legacy=0x7fd3f404f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7fd402200640 1 -- 172.21.15.105:0/3128694399 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f4053c30 con 0x7fd3f404ff50 2024-03-23T07:56:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.339+0000 7f9549600640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95440cd230 0x7f9544157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.340+0000 7f9548c00640 1 -- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95440cd230 msgr2=0x7f9544157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.340+0000 7f9548c00640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95440cd230 0x7f9544157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.340+0000 7fd402200640 1 --2- 172.21.15.105:0/3128694399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f404ff50 0x7fd3f4037b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0695d0 tx=0x7fd3fc0a5610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddb487d177b2eaf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.340+0000 7f9548c00640 1 -- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 msgr2=0x7f9544147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.340+0000 7f9548c00640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f9544147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.341+0000 7f9548c00640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c00ac40 con 0x7f95440710e0 2024-03-23T07:56:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.341+0000 7f9549600640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95440cd230 0x7f9544157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.341+0000 7f9543e00640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f9544147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.341+0000 7fd3fbe00640 1 -- 172.21.15.105:0/3128694399 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0aa830 con 0x7fd3f404ff50 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.341+0000 7fd3fbe00640 1 -- 172.21.15.105:0/3128694399 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc0a8050 con 0x7fd3f404ff50 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.341+0000 7fd3fbe00640 1 -- 172.21.15.105:0/3128694399 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0aac90 con 0x7fd3f404ff50 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.342+0000 7f9548c00640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f95440a74a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f953800c610 tx=0x7f953800cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.343+0000 7f9540c00640 1 -- 172.21.15.105:0/4274406238 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538004220 con 0x7f95440710e0 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.343+0000 7f9540c00640 1 -- 172.21.15.105:0/4274406238 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538004380 con 0x7f95440710e0 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.343+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9544145520 con 0x7f95440710e0 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.343+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9544145ae0 con 0x7f95440710e0 2024-03-23T07:56:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.343+0000 7f9540c00640 1 -- 172.21.15.105:0/4274406238 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538003850 con 0x7f95440710e0 2024-03-23T07:56:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.344+0000 7fd402c00640 1 -- 172.21.15.105:0/3128694399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f404ff50 msgr2=0x7fd3f4037b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.344+0000 7fd402c00640 1 --2- 172.21.15.105:0/3128694399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f404ff50 0x7fd3f4037b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0695d0 tx=0x7fd3fc0a5610 comp rx=0 tx=0).stop 2024-03-23T07:56:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.346+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9544071b10 con 0x7f95440710e0 2024-03-23T07:56:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.347+0000 7f9540c00640 1 -- 172.21.15.105:0/4274406238 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f95380039b0 con 0x7f95440710e0 2024-03-23T07:56:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.348+0000 7fd402c00640 1 -- 172.21.15.105:0/3128694399 shutdown_connections 2024-03-23T07:56:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.348+0000 7fd402c00640 1 --2- 172.21.15.105:0/3128694399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f404ff50 0x7fd3f4037b80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.348+0000 7fd402c00640 1 --2- 172.21.15.105:0/3128694399 >> v2:172.21.15.105:3300/0 conn(0x7fd3f40e32a0 0x7fd3f4052c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.348+0000 7fd402c00640 1 -- 172.21.15.105:0/3128694399 >> 172.21.15.105:0/3128694399 conn(0x7fd3f4009f80 msgr2=0x7fd3f400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.348+0000 7fd402c00640 1 -- 172.21.15.105:0/3128694399 shutdown_connections 2024-03-23T07:56:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.348+0000 7fd402c00640 1 -- 172.21.15.105:0/3128694399 wait complete. 2024-03-23T07:56:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.349+0000 7f9540c00640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952406ac00 0x7f952406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.349+0000 7f9540c00640 1 -- 172.21.15.105:0/4274406238 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f95380892a0 con 0x7f95440710e0 2024-03-23T07:56:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.350+0000 7f9543e00640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952406ac00 0x7f952406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.352+0000 7f9543e00640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952406ac00 0x7f952406d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9530006fd0 tx=0x7f9530008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.353+0000 7fd402c00640 1 Processor -- start 2024-03-23T07:56:49.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.354+0000 7fd402c00640 1 -- start start 2024-03-23T07:56:49.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.355+0000 7fd402c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3f404ff50 0x7fd3f402fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.355+0000 7fd402c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f4053160 0x7fd3f40d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.355+0000 7fd402c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3f40e32a0 0x7fd3f40d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.355+0000 7fd402c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f4032a30 con 0x7fd3f40e32a0 2024-03-23T07:56:49.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.355+0000 7fd402c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f4032d10 con 0x7fd3f4053160 2024-03-23T07:56:49.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.355+0000 7fd402c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f4032ff0 con 0x7fd3f404ff50 2024-03-23T07:56:49.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.356+0000 7fd400e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f4053160 0x7fd3f40d59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.356+0000 7fd400e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f4053160 0x7fd3f40d59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38176/0 (socket says 172.21.15.105:38176) 2024-03-23T07:56:49.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.356+0000 7fd400e00640 1 -- 172.21.15.105:0/2971622506 learned_addr learned my addr 172.21.15.105:0/2971622506 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:49.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.357+0000 7fd400e00640 1 -- 172.21.15.105:0/2971622506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3f404ff50 msgr2=0x7fd3f402fdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T07:56:49.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.357+0000 7fd400e00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3f404ff50 0x7fd3f402fdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.357+0000 7fd400e00640 1 -- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3f40e32a0 msgr2=0x7fd3f40d5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.357+0000 7fd402200640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3f40e32a0 0x7fd3f40d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.357+0000 7fd400e00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3f40e32a0 0x7fd3f40d5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.357+0000 7fd400e00640 1 -- 172.21.15.105:0/2971622506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f40332d0 con 0x7fd3f4053160 2024-03-23T07:56:49.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.357+0000 7fd402200640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3f40e32a0 0x7fd3f40d5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.359+0000 7fd400e00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f4053160 0x7fd3f40d59a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000c780 tx=0x7fd3f000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.359+0000 7fd3f9600640 1 -- 172.21.15.105:0/2971622506 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0005270 con 0x7fd3f4053160 2024-03-23T07:56:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.359+0000 7fd3f9600640 1 -- 172.21.15.105:0/2971622506 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f00053d0 con 0x7fd3f4053160 2024-03-23T07:56:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.359+0000 7fd3f9600640 1 -- 172.21.15.105:0/2971622506 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000a840 con 0x7fd3f4053160 2024-03-23T07:56:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.360+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f40335b0 con 0x7fd3f4053160 2024-03-23T07:56:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.360+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f40304f0 con 0x7fd3f4053160 2024-03-23T07:56:49.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.362+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f40089a0 con 0x7fd3f4053160 2024-03-23T07:56:49.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.364+0000 7f9540c00640 1 -- 172.21.15.105:0/4274406238 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9538054230 con 0x7f95440710e0 2024-03-23T07:56:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.364+0000 7fd3f9600640 1 -- 172.21.15.105:0/2971622506 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd3f000a9a0 con 0x7fd3f4053160 2024-03-23T07:56:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.366+0000 7fd3f9600640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd3e006ac00 0x7fd3e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.367+0000 7fd401800640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd3e006ac00 0x7fd3e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.369+0000 7fd401800640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd3e006ac00 0x7fd3e006d0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009660 tx=0x7fd3ec006b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.369+0000 7fd3f9600640 1 -- 172.21.15.105:0/2971622506 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd3f0057ed0 con 0x7fd3f4053160 2024-03-23T07:56:49.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.386+0000 7fd3f9600640 1 -- 172.21.15.105:0/2971622506 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd3f00542a0 con 0x7fd3f4053160 2024-03-23T07:56:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.663+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f500c005500 con 0x7f50340cb710 2024-03-23T07:56:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.664+0000 7f502be00640 1 -- 172.21.15.105:0/2515177457 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5024057060 con 0x7f50340cb710 2024-03-23T07:56:49.668 INFO:teuthology.orchestra.run.smithi105.stdout:68719476746 2024-03-23T07:56:49.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.668+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5d88005500 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.668+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f501406acd0 msgr2=0x7f501406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.668+0000 7f5039400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f501406acd0 0x7f501406d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5020006fd0 tx=0x7f50200090e0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.669+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50340cb710 msgr2=0x7f50341eb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.669+0000 7f5039400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50340cb710 0x7f50341eb1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f502400c390 tx=0x7f502400c850 comp rx=0 tx=0).stop 2024-03-23T07:56:49.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.669+0000 7f5da9600640 1 -- 172.21.15.105:0/2521128202 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5da0079df0 con 0x7f5dac0cf5f0 2024-03-23T07:56:49.672 INFO:teuthology.orchestra.run.smithi105.stdout:60129542153 2024-03-23T07:56:49.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5d8c06ac60 msgr2=0x7f5d8c06d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5db2e00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5d8c06ac60 0x7f5d8c06d120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0c9220 tx=0x7f5d98008040 comp rx=0 tx=0).stop 2024-03-23T07:56:49.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 shutdown_connections 2024-03-23T07:56:49.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5039400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50340cb710 0x7f50341eb1e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dac0cf5f0 msgr2=0x7f5dac1f1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5db2e00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dac0cf5f0 0x7f5dac1f1b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5da002bf50 tx=0x7f5da0004730 comp rx=0 tx=0).stop 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 shutdown_connections 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5d8c06ac60 0x7f5d8c06d120 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dac160e90 0x7f5dac15f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dac15c820 0x7f5dac15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 --2- 172.21.15.105:0/2521128202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dac0cf5f0 0x7f5dac1f1b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 >> 172.21.15.105:0/2521128202 conn(0x7f5dac06b070 msgr2=0x7f5dac0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5039400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f501406acd0 0x7f501406d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 shutdown_connections 2024-03-23T07:56:49.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5db2e00640 1 -- 172.21.15.105:0/2521128202 wait complete. 2024-03-23T07:56:49.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5039400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50340cefe0 0x7f50340cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5039400640 1 --2- 172.21.15.105:0/2515177457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50340ce360 0x7f50341d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.672+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 >> 172.21.15.105:0/2515177457 conn(0x7f5034089fd0 msgr2=0x7f50340b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 shutdown_connections 2024-03-23T07:56:49.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.673+0000 7f5039400640 1 -- 172.21.15.105:0/2515177457 wait complete. 2024-03-23T07:56:49.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.678+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2e14005500 con 0x7f2e3c160e90 2024-03-23T07:56:49.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.679+0000 7f2e32000640 1 -- 172.21.15.105:0/2411843941 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e280536a0 con 0x7f2e3c160e90 2024-03-23T07:56:49.683 INFO:teuthology.orchestra.run.smithi105.stdout:68719476748 2024-03-23T07:56:49.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.684+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2e1c06ad00 msgr2=0x7f2e1c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.684+0000 7f2e3b200640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2e1c06ad00 0x7f2e1c06d1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0b1410 tx=0x7f2e3400b040 comp rx=0 tx=0).stop 2024-03-23T07:56:49.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.684+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e3c160e90 msgr2=0x7f2e3c14b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.684+0000 7f2e3b200640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e3c160e90 0x7f2e3c14b330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c3c0 tx=0x7f2e2800c880 comp rx=0 tx=0).stop 2024-03-23T07:56:49.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.685+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 shutdown_connections 2024-03-23T07:56:49.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.685+0000 7f2e3b200640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e3c160e90 0x7f2e3c14b330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.685+0000 7f2e3b200640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c15c820 0x7f2e3c152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.685+0000 7f2e3b200640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2e1c06ad00 0x7f2e1c06d1c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.685+0000 7f2e3b200640 1 --2- 172.21.15.105:0/2411843941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e3c0cf5f0 0x7f2e3c14fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.685+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 >> 172.21.15.105:0/2411843941 conn(0x7f2e3c06b070 msgr2=0x7f2e3c0c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.685+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 shutdown_connections 2024-03-23T07:56:49.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.686+0000 7f2e3b200640 1 -- 172.21.15.105:0/2411843941 wait complete. 2024-03-23T07:56:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.691+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f51e404fc00 con 0x7f51d40039c0 2024-03-23T07:56:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.692+0000 7f51dfe00640 1 -- 172.21.15.105:0/1553028133 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51e404fc00 con 0x7f51d40039c0 2024-03-23T07:56:49.693 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-03-23T07:56:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.696+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f51cc06ac80 msgr2=0x7f51cc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.696+0000 7f51f3200640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f51cc06ac80 0x7f51cc06d140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51d8001120 tx=0x7f51d8001a90 comp rx=0 tx=0).stop 2024-03-23T07:56:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.696+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 msgr2=0x7f51e4048e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.696+0000 7f51f3200640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51e4048e80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51e002bb40 tx=0x7f51e00010a0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.697+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 shutdown_connections 2024-03-23T07:56:49.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.697+0000 7f51f3200640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51e40e3280 0x7f51e404e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.697+0000 7f51f3200640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51e40e2e20 0x7f51e404dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.697+0000 7f51f3200640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f51cc06ac80 0x7f51cc06d140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.697+0000 7f51f3200640 1 --2- 172.21.15.105:0/1553028133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51d40039c0 0x7f51e4048e80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.697+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 >> 172.21.15.105:0/1553028133 conn(0x7f51e4053770 msgr2=0x7f51e403ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.698+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 shutdown_connections 2024-03-23T07:56:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.698+0000 7f51f3200640 1 -- 172.21.15.105:0/1553028133 wait complete. 2024-03-23T07:56:49.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.5 2024-03-23T07:56:49.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476746 for osd.1 2024-03-23T07:56:49.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476749 got 68719476748 for osd.0 2024-03-23T07:56:49.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.7 2024-03-23T07:56:49.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.724+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f03180cea20 con 0x7f03180cbc70 2024-03-23T07:56:49.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.725+0000 7f0314c00640 1 -- 172.21.15.105:0/782521722 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f03180cea20 con 0x7f03180cbc70 2024-03-23T07:56:49.725 INFO:teuthology.orchestra.run.smithi105.stdout:68719476746 2024-03-23T07:56:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f02f806ac00 msgr2=0x7f02f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f02f806ac00 0x7f02f806d0c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f030c000f60 tx=0x7f030c0079b0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03180cbc70 msgr2=0x7f0318158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03180cbc70 0x7f0318158440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f030400c430 tx=0x7f030400c8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 shutdown_connections 2024-03-23T07:56:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f031815a960 0x7f031815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03180cf300 0x7f031815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f02f806ac00 0x7f02f806d0c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 --2- 172.21.15.105:0/782521722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03180cbc70 0x7f0318158440 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f030400c430 tx=0x7f030400c8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.727+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 >> 172.21.15.105:0/782521722 conn(0x7f03180b2140 msgr2=0x7f03180affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.728+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 shutdown_connections 2024-03-23T07:56:49.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.728+0000 7f031f400640 1 -- 172.21.15.105:0/782521722 wait complete. 2024-03-23T07:56:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.738+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e54005500 con 0x7f9e780b1f20 2024-03-23T07:56:49.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.739+0000 7f9e74c00640 1 -- 172.21.15.105:0/3766599968 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e6c07a0e0 con 0x7f9e780b1f20 2024-03-23T07:56:49.740 INFO:teuthology.orchestra.run.smithi105.stdout:68719476746 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9e5806ac00 msgr2=0x7f9e5806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9e5806ac00 0x7f9e5806d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9e780b83a0 tx=0x7f9e64009040 comp rx=0 tx=0).stop 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e780b1f20 msgr2=0x7f9e780b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e780b1f20 0x7f9e780b6e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c033000 tx=0x7f9e6c004730 comp rx=0 tx=0).stop 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 shutdown_connections 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9e5806ac00 0x7f9e5806d0c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e7815cc70 0x7f9e780cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e780b4e60 0x7f9e780cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 --2- 172.21.15.105:0/3766599968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e780b1f20 0x7f9e780b6e70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 >> 172.21.15.105:0/3766599968 conn(0x7f9e780cf790 msgr2=0x7f9e780b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 shutdown_connections 2024-03-23T07:56:49.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.742+0000 7f9e7f400640 1 -- 172.21.15.105:0/3766599968 wait complete. 2024-03-23T07:56:49.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476746 for osd.3 2024-03-23T07:56:49.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476746 for osd.2 2024-03-23T07:56:49.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.761+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f95440cc940 con 0x7f95440710e0 2024-03-23T07:56:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.762+0000 7f9540c00640 1 -- 172.21.15.105:0/4274406238 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9538057850 con 0x7f95440710e0 2024-03-23T07:56:49.763 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952406ac00 msgr2=0x7f952406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952406ac00 0x7f952406d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9530006fd0 tx=0x7f9530008040 comp rx=0 tx=0).stop 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 msgr2=0x7f95440a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f95440a74a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f953800c610 tx=0x7f953800cad0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 shutdown_connections 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95440cd230 0x7f9544157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95440b1930 0x7f9544147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952406ac00 0x7f952406d0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 --2- 172.21.15.105:0/4274406238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95440710e0 0x7f95440a74a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 >> 172.21.15.105:0/4274406238 conn(0x7f9544069c90 msgr2=0x7f9544156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 shutdown_connections 2024-03-23T07:56:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.765+0000 7f954a000640 1 -- 172.21.15.105:0/4274406238 wait complete. 2024-03-23T07:56:49.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.6 2024-03-23T07:56:49.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.814+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd3f40304f0 con 0x7fd3f4053160 2024-03-23T07:56:49.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.816+0000 7fd3f9600640 1 -- 172.21.15.105:0/2971622506 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd3f0017030 con 0x7fd3f4053160 2024-03-23T07:56:49.816 INFO:teuthology.orchestra.run.smithi105.stdout:64424509449 2024-03-23T07:56:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd3e006ac00 msgr2=0x7fd3e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd3e006ac00 0x7fd3e006d0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009660 tx=0x7fd3ec006b80 comp rx=0 tx=0).stop 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f4053160 msgr2=0x7fd3f40d59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f4053160 0x7fd3f40d59a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000c780 tx=0x7fd3f000cc40 comp rx=0 tx=0).stop 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 shutdown_connections 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3f40e32a0 0x7fd3f40d5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3f4053160 0x7fd3f40d59a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd3e006ac00 0x7fd3e006d0c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 --2- 172.21.15.105:0/2971622506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3f404ff50 0x7fd3f402fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 >> 172.21.15.105:0/2971622506 conn(0x7fd3f4009f80 msgr2=0x7fd3f40342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 shutdown_connections 2024-03-23T07:56:49.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:49.818+0000 7fd402c00640 1 -- 172.21.15.105:0/2971622506 wait complete. 2024-03-23T07:56:49.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.4 2024-03-23T07:56:50.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-23T07:56:50.705 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-23T07:56:50.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-23T07:56:50.717 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-23T07:56:50.755 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-23T07:56:50.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-23T07:56:50.781 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-23T07:56:50.835 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-23T07:56:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.949+0000 7f43e3200640 1 Processor -- start 2024-03-23T07:56:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.949+0000 7f43e3200640 1 -- start start 2024-03-23T07:56:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.951+0000 7f43e3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f43dc15c7b0 0x7f43dc0b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.951+0000 7f43e3200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43dc0b5730 0x7f43dc0d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.951+0000 7f43e3200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43dc0b6c00 con 0x7f43dc0b5b70 2024-03-23T07:56:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.951+0000 7f43e3200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc0b1fd0 con 0x7f43dc0b5730 2024-03-23T07:56:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.951+0000 7f43e3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc0b5ff0 con 0x7f43dc15c7b0 2024-03-23T07:56:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.952+0000 7f43db400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43dc0b5730 0x7f43dc0d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.952+0000 7f43db400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43dc0b5730 0x7f43dc0d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33566/0 (socket says 172.21.15.105:33566) 2024-03-23T07:56:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.952+0000 7f43db400640 1 -- 172.21.15.105:0/2768451132 learned_addr learned my addr 172.21.15.105:0/2768451132 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.953+0000 7f43db400640 1 -- 172.21.15.105:0/2768451132 >> v2:172.21.15.132:3300/0 conn(0x7f43dc15c7b0 msgr2=0x7f43dc0b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.953+0000 7f43db400640 1 --2- 172.21.15.105:0/2768451132 >> v2:172.21.15.132:3300/0 conn(0x7f43dc15c7b0 0x7f43dc0b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.953+0000 7f43db400640 1 -- 172.21.15.105:0/2768451132 >> v1:172.21.15.105:6789/0 conn(0x7f43dc0b5b70 legacy=0x7f43dc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:50.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.953+0000 7f43db400640 1 -- 172.21.15.105:0/2768451132 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0b62d0 con 0x7f43dc0b5730 2024-03-23T07:56:50.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.955+0000 7f43db400640 1 --2- 172.21.15.105:0/2768451132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc0d0700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0d0440 tx=0x7f43d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4239bb932fadbc05 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.956+0000 7f43daa00640 1 -- 172.21.15.105:0/2768451132 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d402ca80 con 0x7f43dc0b5730 2024-03-23T07:56:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.956+0000 7f43daa00640 1 -- 172.21.15.105:0/2768451132 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d402cbe0 con 0x7f43dc0b5730 2024-03-23T07:56:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.956+0000 7f43daa00640 1 -- 172.21.15.105:0/2768451132 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d40343f0 con 0x7f43dc0b5730 2024-03-23T07:56:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.956+0000 7f383d200640 1 Processor -- start 2024-03-23T07:56:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.957+0000 7f1d18a00640 1 Processor -- start 2024-03-23T07:56:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.957+0000 7f43e3200640 1 -- 172.21.15.105:0/2768451132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 msgr2=0x7f43dc0d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.957+0000 7f43e3200640 1 --2- 172.21.15.105:0/2768451132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc0d0700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0d0440 tx=0x7f43d402bee0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.957+0000 7f43e3200640 1 -- 172.21.15.105:0/2768451132 shutdown_connections 2024-03-23T07:56:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.957+0000 7f43e3200640 1 --2- 172.21.15.105:0/2768451132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc0d0700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.958+0000 7f43e3200640 1 --2- 172.21.15.105:0/2768451132 >> v2:172.21.15.132:3300/0 conn(0x7f43dc15c7b0 0x7f43dc0b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.958+0000 7f43e3200640 1 -- 172.21.15.105:0/2768451132 >> 172.21.15.105:0/2768451132 conn(0x7f43dc0717e0 msgr2=0x7f43dc071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:50.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.957+0000 7f1d18a00640 1 -- start start 2024-03-23T07:56:50.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.957+0000 7f383d200640 1 -- start start 2024-03-23T07:56:50.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.958+0000 7f43e3200640 1 -- 172.21.15.105:0/2768451132 shutdown_connections 2024-03-23T07:56:50.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.958+0000 7f43e3200640 1 -- 172.21.15.105:0/2768451132 wait complete. 2024-03-23T07:56:50.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.959+0000 7f1d18a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d1415cc70 0x7f1d140cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.959+0000 7f1d18a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d140b1f20 0x7f1d140b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.959+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d140b4c90 0x7f1d1406b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.959+0000 7f1d18a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d140b5100 con 0x7f1d1415cc70 2024-03-23T07:56:50.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.959+0000 7f1d18a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d141607d0 con 0x7f1d140b1f20 2024-03-23T07:56:50.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.959+0000 7f1d18a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14160a70 con 0x7f1d140b4c90 2024-03-23T07:56:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.960+0000 7f383d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38380d2380 0x7f38380b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.960+0000 7f383d200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383806b840 con 0x7f383815cd90 2024-03-23T07:56:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.960+0000 7f383d200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383806bb20 con 0x7f38380cd6b0 2024-03-23T07:56:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.960+0000 7f383d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383806c0e0 con 0x7f38380d2380 2024-03-23T07:56:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.960+0000 7f1d12800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d140b4c90 0x7f1d1406b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f1d12800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d140b4c90 0x7f1d1406b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52392/0 (socket says 172.21.15.105:52392) 2024-03-23T07:56:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f1d12800640 1 -- 172.21.15.105:0/1109466185 learned_addr learned my addr 172.21.15.105:0/1109466185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f43e3200640 1 Processor -- start 2024-03-23T07:56:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f1d11400640 1 --2- 172.21.15.105:0/1109466185 >> v2:172.21.15.132:3300/0 conn(0x7f1d140b1f20 0x7f1d140b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f3836800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38380d2380 0x7f38380b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f3836800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38380d2380 0x7f38380b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38208/0 (socket says 172.21.15.105:38208) 2024-03-23T07:56:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f3836800640 1 -- 172.21.15.105:0/2578691803 learned_addr learned my addr 172.21.15.105:0/2578691803 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f43e3200640 1 -- start start 2024-03-23T07:56:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.961+0000 7f1d11e00640 1 --2- 172.21.15.105:0/1109466185 >> v2:172.21.15.105:3300/0 conn(0x7f1d1415cc70 0x7f1d140cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f3836800640 1 -- 172.21.15.105:0/2578691803 >> v1:172.21.15.132:6789/0 conn(0x7f38380cd6b0 legacy=0x7f383806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f3836800640 1 -- 172.21.15.105:0/2578691803 >> v1:172.21.15.105:6789/0 conn(0x7f383815cd90 legacy=0x7f38380cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.963+0000 7f43e3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc15f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.963+0000 7f43e3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43dc0b5b70 0x7f43dc15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.963+0000 7f43e3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43dc15c7b0 0x7f43dc15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.963+0000 7f43e3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0d2730 con 0x7f43dc0b5730 2024-03-23T07:56:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.963+0000 7f43e3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0c6280 con 0x7f43dc0b5b70 2024-03-23T07:56:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.963+0000 7f43e3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc15fcb0 con 0x7f43dc15c7b0 2024-03-23T07:56:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f1d12800640 1 -- 172.21.15.105:0/1109466185 >> v2:172.21.15.132:3300/0 conn(0x7f1d140b1f20 msgr2=0x7f1d140b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f1d12800640 1 --2- 172.21.15.105:0/1109466185 >> v2:172.21.15.132:3300/0 conn(0x7f1d140b1f20 0x7f1d140b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f1d12800640 1 -- 172.21.15.105:0/1109466185 >> v2:172.21.15.105:3300/0 conn(0x7f1d1415cc70 msgr2=0x7f1d140cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f1d12800640 1 --2- 172.21.15.105:0/1109466185 >> v2:172.21.15.105:3300/0 conn(0x7f1d1415cc70 0x7f1d140cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f3836800640 1 -- 172.21.15.105:0/2578691803 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383806be00 con 0x7f38380d2380 2024-03-23T07:56:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.962+0000 7f1d12800640 1 -- 172.21.15.105:0/1109466185 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d14160d50 con 0x7f1d140b4c90 2024-03-23T07:56:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.964+0000 7f43dbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.964+0000 7f43dbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc15f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33580/0 (socket says 172.21.15.105:33580) 2024-03-23T07:56:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.964+0000 7f43dbe00640 1 -- 172.21.15.105:0/2238606813 learned_addr learned my addr 172.21.15.105:0/2238606813 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.964+0000 7f1d12800640 1 --2- 172.21.15.105:0/1109466185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d140b4c90 0x7f1d1406b030 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1d140b8550 tx=0x7f1d0c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b613bde7bbe56505 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.964+0000 7f43db400640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43dc0b5b70 0x7f43dc15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.964+0000 7f43e1200640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43dc15c7b0 0x7f43dc15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f3836800640 1 --2- 172.21.15.105:0/2578691803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f38380b8490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85252a78817c4473 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f1d10a00640 1 -- 172.21.15.105:0/1109466185 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c030800 con 0x7f1d140b4c90 2024-03-23T07:56:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f1d10a00640 1 -- 172.21.15.105:0/1109466185 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c030960 con 0x7f1d140b4c90 2024-03-23T07:56:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f1d10a00640 1 -- 172.21.15.105:0/1109466185 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c030d30 con 0x7f1d140b4c90 2024-03-23T07:56:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f43db400640 1 -- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43dc15c7b0 msgr2=0x7f43dc15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f43db400640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43dc15c7b0 0x7f43dc15e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f43db400640 1 -- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 msgr2=0x7f43dc15f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f43db400640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc15f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f43db400640 1 -- 172.21.15.105:0/2238606813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d4009b30 con 0x7f43dc0b5b70 2024-03-23T07:56:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f3834a00640 1 -- 172.21.15.105:0/2578691803 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383002ca80 con 0x7f38380d2380 2024-03-23T07:56:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f43db400640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43dc0b5b70 0x7f43dc15e070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f43d4032000 tx=0x7f43d4004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f1d18a00640 1 -- 172.21.15.105:0/1109466185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d140b4c90 msgr2=0x7f1d1406b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f1d18a00640 1 --2- 172.21.15.105:0/1109466185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d140b4c90 0x7f1d1406b030 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1d140b8550 tx=0x7f1d0c02b940 comp rx=0 tx=0).stop 2024-03-23T07:56:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f3834a00640 1 -- 172.21.15.105:0/2578691803 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383002cbe0 con 0x7f38380d2380 2024-03-23T07:56:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.965+0000 7f3834a00640 1 -- 172.21.15.105:0/2578691803 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300343f0 con 0x7f38380d2380 2024-03-23T07:56:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f383d200640 1 -- 172.21.15.105:0/2578691803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 msgr2=0x7f38380b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.967+0000 7f383d200640 1 --2- 172.21.15.105:0/2578691803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f38380b8490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383002bee0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f1d18a00640 1 -- 172.21.15.105:0/1109466185 shutdown_connections 2024-03-23T07:56:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f1d18a00640 1 --2- 172.21.15.105:0/1109466185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d140b4c90 0x7f1d1406b030 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f1d18a00640 1 --2- 172.21.15.105:0/1109466185 >> v2:172.21.15.132:3300/0 conn(0x7f1d140b1f20 0x7f1d140b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f1d18a00640 1 --2- 172.21.15.105:0/1109466185 >> v2:172.21.15.105:3300/0 conn(0x7f1d1415cc70 0x7f1d140cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.966+0000 7f1d18a00640 1 -- 172.21.15.105:0/1109466185 >> 172.21.15.105:0/1109466185 conn(0x7f1d140cf790 msgr2=0x7f1d140cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.967+0000 7f1d18a00640 1 -- 172.21.15.105:0/1109466185 shutdown_connections 2024-03-23T07:56:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.967+0000 7f43cfe00640 1 -- 172.21.15.105:0/2238606813 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d403e450 con 0x7f43dc0b5b70 2024-03-23T07:56:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.967+0000 7f43cfe00640 1 -- 172.21.15.105:0/2238606813 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d403e5b0 con 0x7f43dc0b5b70 2024-03-23T07:56:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.967+0000 7f43cfe00640 1 -- 172.21.15.105:0/2238606813 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d403e920 con 0x7f43dc0b5b70 2024-03-23T07:56:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.967+0000 7f383d200640 1 -- 172.21.15.105:0/2578691803 shutdown_connections 2024-03-23T07:56:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.968+0000 7f383d200640 1 --2- 172.21.15.105:0/2578691803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f38380b8490 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.968+0000 7f383d200640 1 -- 172.21.15.105:0/2578691803 >> 172.21.15.105:0/2578691803 conn(0x7f38380cf780 msgr2=0x7f38380cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.968+0000 7f383d200640 1 -- 172.21.15.105:0/2578691803 shutdown_connections 2024-03-23T07:56:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.968+0000 7f383d200640 1 -- 172.21.15.105:0/2578691803 wait complete. 2024-03-23T07:56:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.968+0000 7f43e3200640 1 -- 172.21.15.105:0/2238606813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc0cb7b0 con 0x7f43dc0b5b70 2024-03-23T07:56:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.969+0000 7f43e3200640 1 -- 172.21.15.105:0/2238606813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc0cc050 con 0x7f43dc0b5b70 2024-03-23T07:56:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.967+0000 7f1d18a00640 1 -- 172.21.15.105:0/1109466185 wait complete. 2024-03-23T07:56:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.969+0000 7f1d18a00640 1 Processor -- start 2024-03-23T07:56:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.970+0000 7f1d18a00640 1 -- start start 2024-03-23T07:56:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.970+0000 7fbf6fe00640 1 Processor -- start 2024-03-23T07:56:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.970+0000 7fbf6fe00640 1 -- start start 2024-03-23T07:56:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.971+0000 7f43e3200640 1 -- 172.21.15.105:0/2238606813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc0c6410 con 0x7f43dc0b5b70 2024-03-23T07:56:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.971+0000 7f383d200640 1 Processor -- start 2024-03-23T07:56:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f383d200640 1 -- start start 2024-03-23T07:56:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7fbf6fe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf700cd830 0x7fbf70160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7fbf6fe00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf701613d0 con 0x7fbf7015c820 2024-03-23T07:56:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7fbf6fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf700ccd90 con 0x7fbf700cf7c0 2024-03-23T07:56:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7fbf6fe00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf700b7fc0 con 0x7fbf700cd830 2024-03-23T07:56:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.973+0000 7fbf6f400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf700cd830 0x7fbf70160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.973+0000 7fbf6f400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf700cd830 0x7fbf70160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33612/0 (socket says 172.21.15.105:33612) 2024-03-23T07:56:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.973+0000 7fbf6f400640 1 -- 172.21.15.105:0/425412017 learned_addr learned my addr 172.21.15.105:0/425412017 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7fbf6f400640 1 -- 172.21.15.105:0/425412017 >> v1:172.21.15.132:6789/0 conn(0x7fbf700cf7c0 legacy=0x7fbf700d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7fbf6f400640 1 -- 172.21.15.105:0/425412017 >> v1:172.21.15.105:6789/0 conn(0x7fbf7015c820 legacy=0x7fbf700cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7fbf6f400640 1 -- 172.21.15.105:0/425412017 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700b82a0 con 0x7fbf700cd830 2024-03-23T07:56:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f383d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38380cd6b0 0x7f3838151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f383d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f3838152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f383d200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f383815cd90 0x7f3838154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f43cfe00640 1 -- 172.21.15.105:0/2238606813 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f43d402cd50 con 0x7f43dc0b5b70 2024-03-23T07:56:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f383d200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380cb500 con 0x7f383815cd90 2024-03-23T07:56:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f383d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838152920 con 0x7f38380d2380 2024-03-23T07:56:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.975+0000 7f43cfe00640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f43bc06acd0 0x7f43bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f383d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3838153cc0 con 0x7f38380cd6b0 2024-03-23T07:56:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d140b1f20 0x7f1d1415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d140b4c90 0x7f1d140b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d1415cc70 0x7f1d140b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f1d18a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140ccb10 con 0x7f1d140b4c90 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f1d18a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1415a6b0 con 0x7f1d140b1f20 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f1d18a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d141600e0 con 0x7f1d1415cc70 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.972+0000 7f1d11e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d140b1f20 0x7f1d1415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.973+0000 7f1d11e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d140b1f20 0x7f1d1415fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38226/0 (socket says 172.21.15.105:38226) 2024-03-23T07:56:50.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.973+0000 7f1d11e00640 1 -- 172.21.15.105:0/914428654 learned_addr learned my addr 172.21.15.105:0/914428654 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7fbf6f400640 1 --2- 172.21.15.105:0/425412017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf70160e90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e50 tx=0x7fbf6802c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62725fb17c092b09 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.973+0000 7f1d11e00640 1 -- 172.21.15.105:0/914428654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d1415cc70 msgr2=0x7f1d140b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.973+0000 7f1d11e00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d1415cc70 0x7f1d140b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f1d11e00640 1 -- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d140b4c90 msgr2=0x7f1d140b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f1d11e00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d140b4c90 0x7f1d140b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.974+0000 7f1d11e00640 1 -- 172.21.15.105:0/914428654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c0094d0 con 0x7f1d140b1f20 2024-03-23T07:56:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.975+0000 7f1d11e00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d140b1f20 0x7f1d1415fba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c820 tx=0x7f1cfc00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7f43cfe00640 1 -- 172.21.15.105:0/2238606813 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f43d40ac1c0 con 0x7f43dc0b5b70 2024-03-23T07:56:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7fbf6d600640 1 -- 172.21.15.105:0/425412017 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6802cc40 con 0x7fbf700cd830 2024-03-23T07:56:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7f3835e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38380cd6b0 0x7f3838151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7f3835e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38380cd6b0 0x7f3838151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52410/0 (socket says 172.21.15.105:52410) 2024-03-23T07:56:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7f3835e00640 1 -- 172.21.15.105:0/2392026681 learned_addr learned my addr 172.21.15.105:0/2392026681 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7fbf6d600640 1 -- 172.21.15.105:0/425412017 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6802cda0 con 0x7fbf700cd830 2024-03-23T07:56:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7f3836800640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f383815cd90 0x7f3838154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7f43dbe00640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f43bc06acd0 0x7f43bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.976+0000 7fbf6d600640 1 -- 172.21.15.105:0/425412017 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68034610 con 0x7fbf700cd830 2024-03-23T07:56:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7f3836800640 1 -- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38380cd6b0 msgr2=0x7f3838151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7f3836800640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38380cd6b0 0x7f3838151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7f3836800640 1 -- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 msgr2=0x7f3838152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7f3835400640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f3838152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7fbf6fe00640 1 -- 172.21.15.105:0/425412017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 msgr2=0x7fbf70160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/425412017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf70160e90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e50 tx=0x7fbf6802c0a0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7f3836800640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f3838152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7fbf6fe00640 1 -- 172.21.15.105:0/425412017 shutdown_connections 2024-03-23T07:56:50.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/425412017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf70160e90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7f43dbe00640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f43bc06acd0 0x7f43bc06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0c9210 tx=0x7f43c8006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7fbf6fe00640 1 -- 172.21.15.105:0/425412017 >> 172.21.15.105:0/425412017 conn(0x7fbf7006b070 msgr2=0x7fbf7006b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:50.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7fbf6fe00640 1 -- 172.21.15.105:0/425412017 shutdown_connections 2024-03-23T07:56:50.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.977+0000 7f3836800640 1 -- 172.21.15.105:0/2392026681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830009b30 con 0x7f383815cd90 2024-03-23T07:56:50.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7f3835400640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f3838152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:50.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7f3836800640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f383815cd90 0x7f3838154f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3830032000 tx=0x7f3830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.978+0000 7fbf6fe00640 1 -- 172.21.15.105:0/425412017 wait complete. 2024-03-23T07:56:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.979+0000 7f382e000640 1 -- 172.21.15.105:0/2392026681 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383003e450 con 0x7f383815cd90 2024-03-23T07:56:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.979+0000 7f382e000640 1 -- 172.21.15.105:0/2392026681 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383003e5d0 con 0x7f383815cd90 2024-03-23T07:56:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.979+0000 7f382e000640 1 -- 172.21.15.105:0/2392026681 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383003e780 con 0x7f383815cd90 2024-03-23T07:56:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.980+0000 7fbf6fe00640 1 Processor -- start 2024-03-23T07:56:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.979+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3838153fa0 con 0x7f383815cd90 2024-03-23T07:56:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.979+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381532f0 con 0x7f383815cd90 2024-03-23T07:56:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.981+0000 7fbf6fe00640 1 -- start start 2024-03-23T07:56:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.981+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3838153b20 con 0x7f383815cd90 2024-03-23T07:56:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.982+0000 7f382e000640 1 -- 172.21.15.105:0/2392026681 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f383002cd50 con 0x7f383815cd90 2024-03-23T07:56:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf700c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7f382e000640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f381806acd0 0x7f381806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7f382e000640 1 -- 172.21.15.105:0/2392026681 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38300abde0 con 0x7f383815cd90 2024-03-23T07:56:50.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7f3835e00640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f381806acd0 0x7f381806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf700cf7c0 0x7fbf70151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf7015c820 0x7fbf700b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700c7270 con 0x7fbf700cd830 2024-03-23T07:56:50.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701520e0 con 0x7fbf7015c820 2024-03-23T07:56:50.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.983+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700c6660 con 0x7fbf700cf7c0 2024-03-23T07:56:50.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf700c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.985+0000 7f3835e00640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f381806acd0 0x7f381806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38380b3c90 tx=0x7f3828009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf700c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33644/0 (socket says 172.21.15.105:33644) 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 -- 172.21.15.105:0/2943074923 learned_addr learned my addr 172.21.15.105:0/2943074923 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6e000640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf700cf7c0 0x7fbf70151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6f400640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf7015c820 0x7fbf700b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 -- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf700cf7c0 msgr2=0x7fbf70151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf700cf7c0 0x7fbf70151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 -- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf7015c820 msgr2=0x7fbf700b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf7015c820 0x7fbf700b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:50.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.984+0000 7fbf6ea00640 1 -- 172.21.15.105:0/2943074923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf68009b30 con 0x7fbf700cd830 2024-03-23T07:56:50.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.985+0000 7fbf6ea00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf700c7700 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c450 tx=0x7fbf6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:50.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.988+0000 7f43cfe00640 1 -- 172.21.15.105:0/2238606813 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f43d4077200 con 0x7f43dc0b5b70 2024-03-23T07:56:50.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.989+0000 7f1d0a000640 1 -- 172.21.15.105:0/914428654 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc004020 con 0x7f1d140b1f20 2024-03-23T07:56:50.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.989+0000 7f1d0a000640 1 -- 172.21.15.105:0/914428654 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc004180 con 0x7f1d140b1f20 2024-03-23T07:56:50.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.989+0000 7f1d0a000640 1 -- 172.21.15.105:0/914428654 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc007830 con 0x7f1d140b1f20 2024-03-23T07:56:50.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.989+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d140cc550 con 0x7f1d140b1f20 2024-03-23T07:56:50.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.989+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1415d1f0 con 0x7f1d140b1f20 2024-03-23T07:56:50.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.990+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1415d050 con 0x7f1d140b1f20 2024-03-23T07:56:50.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.993+0000 7f1d0a000640 1 -- 172.21.15.105:0/914428654 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1cfc007990 con 0x7f1d140b1f20 2024-03-23T07:56:50.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.994+0000 7fbf66a00640 1 -- 172.21.15.105:0/2943074923 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600052d0 con 0x7fbf700cd830 2024-03-23T07:56:50.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.994+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf700c6940 con 0x7fbf700cd830 2024-03-23T07:56:50.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.994+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf700c6f00 con 0x7fbf700cd830 2024-03-23T07:56:50.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.994+0000 7f1d0a000640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cf006acd0 0x7f1cf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:50.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.995+0000 7f1d0a000640 1 -- 172.21.15.105:0/914428654 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1cfc0893f0 con 0x7f1d140b1f20 2024-03-23T07:56:50.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.996+0000 7f1d11400640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cf006acd0 0x7f1cf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:50.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.996+0000 7fbf66a00640 1 -- 172.21.15.105:0/2943074923 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60005430 con 0x7fbf700cd830 2024-03-23T07:56:50.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.996+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7004e030 con 0x7fbf700cd830 2024-03-23T07:56:50.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.996+0000 7fbf66a00640 1 -- 172.21.15.105:0/2943074923 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60012870 con 0x7fbf700cd830 2024-03-23T07:56:50.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.997+0000 7f382e000640 1 -- 172.21.15.105:0/2392026681 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3838153b20 con 0x7f383815cd90 2024-03-23T07:56:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.997+0000 7f1d11400640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cf006acd0 0x7f1cf006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d14071390 tx=0x7f1d0400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:50.999+0000 7fbf66a00640 1 -- 172.21.15.105:0/2943074923 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbf60012a50 con 0x7fbf700cd830 2024-03-23T07:56:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.002+0000 7fbf66a00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf5006ac30 0x7fbf5006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.002+0000 7fbf66a00640 1 -- 172.21.15.105:0/2943074923 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbf60088fd0 con 0x7fbf700cd830 2024-03-23T07:56:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.003+0000 7fbf6e000640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf5006ac30 0x7fbf5006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.005+0000 7fbf6e000640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf5006ac30 0x7fbf5006d0f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf700b4930 tx=0x7fbf5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.006+0000 7f1d0a000640 1 -- 172.21.15.105:0/914428654 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1cfc054350 con 0x7f1d140b1f20 2024-03-23T07:56:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.009+0000 7fbf66a00640 1 -- 172.21.15.105:0/2943074923 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbf60053ec0 con 0x7fbf700cd830 2024-03-23T07:56:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.057+0000 7f7362a00640 1 Processor -- start 2024-03-23T07:56:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.057+0000 7f9b03200640 1 Processor -- start 2024-03-23T07:56:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.057+0000 7f7362a00640 1 -- start start 2024-03-23T07:56:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.059+0000 7f7362a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f735c1588b0 0x7f735c070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.058+0000 7f9b03200640 1 -- start start 2024-03-23T07:56:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.060+0000 7f9b03200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9afc0df3f0 0x7f9afc053230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.060+0000 7f9b03200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc006870 con 0x7f9afc0392a0 2024-03-23T07:56:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.060+0000 7f9b03200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc006ac0 con 0x7f9afc04fa60 2024-03-23T07:56:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.060+0000 7f9b03200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc007740 con 0x7f9afc0df3f0 2024-03-23T07:56:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.060+0000 7f9b01e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f9afc04fa60 0x7f9afc04fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.105:45590/0 (socket says 172.21.15.105:45590) 2024-03-23T07:56:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.060+0000 7f9b01e00640 1 -- 172.21.15.105:0/3327149674 learned_addr learned my addr 172.21.15.105:0/3327149674 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.061+0000 7f7362a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c15d220 con 0x7f735c158c90 2024-03-23T07:56:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.061+0000 7f7362a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c1591c0 con 0x7f735c15cc00 2024-03-23T07:56:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.061+0000 7f7360a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f735c15cc00 0x7f735c0b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.105:45600/0 (socket says 172.21.15.105:45600) 2024-03-23T07:56:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.061+0000 7f7360a00640 1 -- 172.21.15.105:0/2603348754 learned_addr learned my addr 172.21.15.105:0/2603348754 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.061+0000 7f7362a00640 1 -- 172.21.15.105:0/2603348754 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c15d040 con 0x7f735c1588b0 2024-03-23T07:56:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.061+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436527174 0 0) 0x7f9afc006ac0 con 0x7f9afc04fa60 2024-03-23T07:56:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.061+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae8003520 con 0x7f9afc04fa60 2024-03-23T07:56:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f735be00640 1 --2- 172.21.15.105:0/2603348754 >> v2:172.21.15.105:3300/0 conn(0x7f735c1588b0 0x7f735c070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114279261 0 0) 0x7f735c1591c0 con 0x7f735c15cc00 2024-03-23T07:56:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7340003520 con 0x7f735c15cc00 2024-03-23T07:56:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f9b01400640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9afc0df3f0 0x7f9afc053230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3136004809 0 0) 0x7f9ae8003520 con 0x7f9afc04fa60 2024-03-23T07:56:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9afc0df3f0 msgr2=0x7f9afc053230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f9b00a00640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9afc0df3f0 0x7f9afc053230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 >> v1:172.21.15.105:6789/0 conn(0x7f9afc0392a0 legacy=0x7f9afc053770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.062+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc006da0 con 0x7f9afc04fa60 2024-03-23T07:56:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3904492657 0 0) 0x7f7340003520 con 0x7f735c15cc00 2024-03-23T07:56:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 >> v2:172.21.15.105:3300/0 conn(0x7f735c1588b0 msgr2=0x7f735c070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f9af4001650 con 0x7f9afc04fa60 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 >> v1:172.21.15.132:6789/0 conn(0x7f9afc04fa60 legacy=0x7f9afc04fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae8003990 0x7f9ae8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80047c0 0x7f9ae8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80052a0 0x7f9ae8007960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc007740 con 0x7f9ae80047c0 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80081e0 con 0x7f9ae8003990 2024-03-23T07:56:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80083a0 con 0x7f9ae80052a0 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 --2- 172.21.15.105:0/2603348754 >> v2:172.21.15.105:3300/0 conn(0x7f735c1588b0 0x7f735c070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 >> v1:172.21.15.105:6789/0 conn(0x7f735c158c90 legacy=0x7f735c15c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c1594a0 con 0x7f735c15cc00 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f734c001670 con 0x7f735c15cc00 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 >> v1:172.21.15.132:6789/0 conn(0x7f735c15cc00 legacy=0x7f735c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.063+0000 7f735aa00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7340003970 0x7f7340003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f735aa00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73400047c0 0x7f7340004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f735aa00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340005380 0x7f734000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c15d040 con 0x7f7340005380 2024-03-23T07:56:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734000c210 con 0x7f7340003970 2024-03-23T07:56:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734000c3d0 con 0x7f73400047c0 2024-03-23T07:56:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f735be00640 1 --2- 172.21.15.105:0/2603348754 >> v2:172.21.15.105:3300/0 conn(0x7f735c1588b0 0x7f735c070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f9b01400640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9afc0df3f0 0x7f9afc053230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f9b02800640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80052a0 0x7f9ae8007960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.064+0000 7f9b01e00640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae8003990 0x7f9ae8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.065+0000 7f735be00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7340003970 0x7f7340003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.065+0000 7f735b400640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73400047c0 0x7f7340004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.065+0000 7f7360a00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340005380 0x7f734000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.065+0000 7f9b01400640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80047c0 0x7f9ae8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.066+0000 7f735b400640 1 -- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7340003970 msgr2=0x7f7340003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.066+0000 7f735b400640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7340003970 0x7f7340003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.066+0000 7f735b400640 1 -- 172.21.15.105:0/2603348754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340005380 msgr2=0x7f734000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.067+0000 7f9b01400640 1 -- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80052a0 msgr2=0x7f9ae8007960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.067+0000 7f9b01400640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80052a0 0x7f9ae8007960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.066+0000 7f735b400640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340005380 0x7f734000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.066+0000 7f735b400640 1 -- 172.21.15.105:0/2603348754 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c1594a0 con 0x7f73400047c0 2024-03-23T07:56:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.066+0000 7f7360a00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340005380 0x7f734000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.066+0000 7f7360a00640 1 -- 172.21.15.105:0/2603348754 reap_dead start 2024-03-23T07:56:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.067+0000 7f9b01400640 1 -- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae8003990 msgr2=0x7f9ae8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.067+0000 7f9b01400640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae8003990 0x7f9ae8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.068+0000 7f9b01400640 1 -- 172.21.15.105:0/3327149674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8003520 con 0x7f9ae80047c0 2024-03-23T07:56:51.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.068+0000 7f735b400640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73400047c0 0x7f7340004c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7348001750 tx=0x7f73400057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e207fd036cdb8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.068+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480052d0 con 0x7f73400047c0 2024-03-23T07:56:51.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.069+0000 7f735aa00640 1 -- 172.21.15.105:0/2603348754 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734802e050 con 0x7f73400047c0 2024-03-23T07:56:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.068+0000 7f9b02800640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80052a0 0x7f9ae8007960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.069+0000 7f9b01e00640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae8003990 0x7f9ae8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.070+0000 7f9b01400640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80047c0 0x7f9ae8004c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9afc008530 tx=0x7f9ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0b1ca3d6e0ad5f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.070+0000 7f7362a00640 1 -- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73400047c0 msgr2=0x7f7340004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.070+0000 7f7362a00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73400047c0 0x7f7340004c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7348001750 tx=0x7f73400057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.071+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0032620 con 0x7f9ae80047c0 2024-03-23T07:56:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.071+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00327a0 con 0x7f9ae80047c0 2024-03-23T07:56:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.071+0000 7f9b00a00640 1 -- 172.21.15.105:0/3327149674 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0032b50 con 0x7f9ae80047c0 2024-03-23T07:56:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.072+0000 7f7362a00640 1 -- 172.21.15.105:0/2603348754 shutdown_connections 2024-03-23T07:56:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.072+0000 7f7362a00640 1 --2- 172.21.15.105:0/2603348754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73400047c0 0x7f7340004c40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.072+0000 7f7362a00640 1 -- 172.21.15.105:0/2603348754 >> 172.21.15.105:0/2603348754 conn(0x7f735c069c90 msgr2=0x7f735c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.072+0000 7f7362a00640 1 -- 172.21.15.105:0/2603348754 shutdown_connections 2024-03-23T07:56:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.072+0000 7f9b03200640 1 -- 172.21.15.105:0/3327149674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80047c0 msgr2=0x7f9ae8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.074+0000 7f9b03200640 1 --2- 172.21.15.105:0/3327149674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80047c0 0x7f9ae8004c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9afc008530 tx=0x7f9ae80057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.074+0000 7f9b02800640 1 -- 172.21.15.105:0/3327149674 reap_dead start 2024-03-23T07:56:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.072+0000 7f7362a00640 1 -- 172.21.15.105:0/2603348754 wait complete. 2024-03-23T07:56:51.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.074+0000 7f9b03200640 1 -- 172.21.15.105:0/3327149674 shutdown_connections 2024-03-23T07:56:51.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.074+0000 7f9b03200640 1 -- 172.21.15.105:0/3327149674 >> 172.21.15.105:0/3327149674 conn(0x7f9afc009f80 msgr2=0x7f9afc00a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.074+0000 7f9b03200640 1 -- 172.21.15.105:0/3327149674 shutdown_connections 2024-03-23T07:56:51.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.074+0000 7f9b03200640 1 -- 172.21.15.105:0/3327149674 wait complete. 2024-03-23T07:56:51.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.076+0000 7f7362a00640 1 Processor -- start 2024-03-23T07:56:51.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.076+0000 7f7362a00640 1 -- start start 2024-03-23T07:56:51.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.077+0000 7f9b03200640 1 Processor -- start 2024-03-23T07:56:51.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.077+0000 7f9b03200640 1 -- start start 2024-03-23T07:56:51.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f7362a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73400047c0 0x7f735c14d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f7362a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c156130 0x7f735c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f7362a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b2840 0x7f735c1f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f7362a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0ae310 con 0x7f73400047c0 2024-03-23T07:56:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f7362a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0b2d00 con 0x7f735c156130 2024-03-23T07:56:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f7362a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0b2fe0 con 0x7f735c0b2840 2024-03-23T07:56:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f735be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73400047c0 0x7f735c14d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.079+0000 7f9b03200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80047c0 0x7f9afc04e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b03200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc0d1e90 0x7f9afc04bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b03200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0d2310 0x7f9afc04c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b03200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0e0170 con 0x7f9afc0d2310 2024-03-23T07:56:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b03200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc04e9f0 con 0x7f9afc0d1e90 2024-03-23T07:56:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b03200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc04a560 con 0x7f9ae80047c0 2024-03-23T07:56:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b01e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80047c0 0x7f9afc04e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b01e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80047c0 0x7f9afc04e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52460/0 (socket says 172.21.15.105:52460) 2024-03-23T07:56:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f9b01e00640 1 -- 172.21.15.105:0/696181165 learned_addr learned my addr 172.21.15.105:0/696181165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f7360a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b2840 0x7f735c1f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73400047c0 0x7f735c14d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33686/0 (socket says 172.21.15.105:33686) 2024-03-23T07:56:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c156130 0x7f735c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735be00640 1 -- 172.21.15.105:0/2060773119 learned_addr learned my addr 172.21.15.105:0/2060773119 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b01400640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc0d1e90 0x7f9afc04bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735be00640 1 -- 172.21.15.105:0/2060773119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b2840 msgr2=0x7f735c1f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735be00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b2840 0x7f735c1f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735be00640 1 -- 172.21.15.105:0/2060773119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c156130 msgr2=0x7f735c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735be00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c156130 0x7f735c0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.080+0000 7f735be00640 1 -- 172.21.15.105:0/2060773119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c159e20 con 0x7f73400047c0 2024-03-23T07:56:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b01e00640 1 -- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc0d1e90 msgr2=0x7f9afc04bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b02800640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0d2310 0x7f9afc04c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b01e00640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc0d1e90 0x7f9afc04bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b01e00640 1 -- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0d2310 msgr2=0x7f9afc04c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b01e00640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0d2310 0x7f9afc04c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b01e00640 1 -- 172.21.15.105:0/696181165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc0df7d0 con 0x7f9ae80047c0 2024-03-23T07:56:51.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f735be00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73400047c0 0x7f735c14d2a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7354007fd0 tx=0x7f73540085a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.081+0000 7f9b02800640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0d2310 0x7f9afc04c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:51.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.082+0000 7f9b01400640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc0d1e90 0x7f9afc04bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:51.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.082+0000 7f7353e00640 1 -- 172.21.15.105:0/2060773119 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73540157e0 con 0x7f73400047c0 2024-03-23T07:56:51.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.082+0000 7f7353e00640 1 -- 172.21.15.105:0/2060773119 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354015940 con 0x7f73400047c0 2024-03-23T07:56:51.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.082+0000 7f7353e00640 1 -- 172.21.15.105:0/2060773119 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354015cb0 con 0x7f73400047c0 2024-03-23T07:56:51.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.082+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c0ae560 con 0x7f73400047c0 2024-03-23T07:56:51.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.082+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c14d7e0 con 0x7f73400047c0 2024-03-23T07:56:51.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.084+0000 7f9b01e00640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80047c0 0x7f9afc04e4b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9af4001640 tx=0x7f9af400d640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.084+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c0b31a0 con 0x7f73400047c0 2024-03-23T07:56:51.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.084+0000 7f9afa000640 1 -- 172.21.15.105:0/696181165 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4012470 con 0x7f9ae80047c0 2024-03-23T07:56:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.085+0000 7f9afa000640 1 -- 172.21.15.105:0/696181165 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af40125d0 con 0x7f9ae80047c0 2024-03-23T07:56:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.085+0000 7f9afa000640 1 -- 172.21.15.105:0/696181165 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4012940 con 0x7f9ae80047c0 2024-03-23T07:56:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.085+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc0dfab0 con 0x7f9ae80047c0 2024-03-23T07:56:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.085+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc04c5e0 con 0x7f9ae80047c0 2024-03-23T07:56:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.088+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc04f8d0 con 0x7f9ae80047c0 2024-03-23T07:56:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.091+0000 7f9afa000640 1 -- 172.21.15.105:0/696181165 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9af4012aa0 con 0x7f9ae80047c0 2024-03-23T07:56:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.092+0000 7f9afa000640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ae406ac00 0x7f9ae406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.092+0000 7f9afa000640 1 -- 172.21.15.105:0/696181165 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9af408a480 con 0x7f9ae80047c0 2024-03-23T07:56:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.092+0000 7f7353e00640 1 -- 172.21.15.105:0/2060773119 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7354003990 con 0x7f73400047c0 2024-03-23T07:56:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.094+0000 7f7353e00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f733c06ac00 0x7f733c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.094+0000 7f7353e00640 1 -- 172.21.15.105:0/2060773119 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f735408b5b0 con 0x7f73400047c0 2024-03-23T07:56:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.099+0000 7f9b01400640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ae406ac00 0x7f9ae406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.099+0000 7f735b400640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f733c06ac00 0x7f733c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.100+0000 7f7353e00640 1 -- 172.21.15.105:0/2060773119 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f73540563f0 con 0x7f73400047c0 2024-03-23T07:56:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.101+0000 7f9b01400640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ae406ac00 0x7f9ae406d0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9af0036000 tx=0x7f9af0031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.101+0000 7f735b400640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f733c06ac00 0x7f733c06d0c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7348033000 tx=0x7f7348009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.102+0000 7fa2b2e00640 1 Processor -- start 2024-03-23T07:56:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.102+0000 7fa2b2e00640 1 -- start start 2024-03-23T07:56:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.104+0000 7fa2b2e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2ac070170 0x7fa2ac0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.105+0000 7fa2b2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac15c4f0 0x7fa2ac15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.105+0000 7f9afa000640 1 -- 172.21.15.105:0/696181165 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9af40553e0 con 0x7f9ae80047c0 2024-03-23T07:56:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.105+0000 7fa2b2e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac15d220 con 0x7fa2ac06a780 2024-03-23T07:56:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.105+0000 7fa2b2e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac158fc0 con 0x7fa2ac070170 2024-03-23T07:56:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.105+0000 7fa2b2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac070b20 con 0x7fa2ac15c4f0 2024-03-23T07:56:51.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.106+0000 7fa2abe00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa2ac06a780 0x7fa2ac1588b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.105:45606/0 (socket says 172.21.15.105:45606) 2024-03-23T07:56:51.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.106+0000 7fa2abe00640 1 -- 172.21.15.105:0/657791285 learned_addr learned my addr 172.21.15.105:0/657791285 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:51.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.106+0000 7fa2ab400640 1 --2- 172.21.15.105:0/657791285 >> v2:172.21.15.105:3300/0 conn(0x7fa2ac070170 0x7fa2ac0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.106+0000 7fa2b0e00640 1 --2- 172.21.15.105:0/657791285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac15c4f0 0x7fa2ac15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.107+0000 7fa2ab400640 1 -- 172.21.15.105:0/657791285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac15c4f0 msgr2=0x7fa2ac15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.107+0000 7fa2ab400640 1 --2- 172.21.15.105:0/657791285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac15c4f0 0x7fa2ac15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.107+0000 7fa2ab400640 1 -- 172.21.15.105:0/657791285 >> v1:172.21.15.132:6789/0 conn(0x7fa2ac06a780 legacy=0x7fa2ac1588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.107+0000 7fa2ab400640 1 -- 172.21.15.105:0/657791285 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac1591c0 con 0x7fa2ac070170 2024-03-23T07:56:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.109+0000 7fa2ab400640 1 --2- 172.21.15.105:0/657791285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac070170 0x7fa2ac0705e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2980097d0 tx=0x7fa29802b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738508c0f7f10987 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.110+0000 7fa2aaa00640 1 -- 172.21.15.105:0/657791285 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298030710 con 0x7fa2ac070170 2024-03-23T07:56:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.111+0000 7fa2aaa00640 1 -- 172.21.15.105:0/657791285 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29802e050 con 0x7fa2ac070170 2024-03-23T07:56:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.111+0000 7fa2aaa00640 1 -- 172.21.15.105:0/657791285 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298030710 con 0x7fa2ac070170 2024-03-23T07:56:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.112+0000 7fa2b2e00640 1 -- 172.21.15.105:0/657791285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac070170 msgr2=0x7fa2ac0705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.112+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/657791285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac070170 0x7fa2ac0705e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2980097d0 tx=0x7fa29802b5d0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.113+0000 7fa2b2e00640 1 -- 172.21.15.105:0/657791285 shutdown_connections 2024-03-23T07:56:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.113+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/657791285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac15c4f0 0x7fa2ac15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.113+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/657791285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac070170 0x7fa2ac0705e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.113+0000 7fa2b2e00640 1 -- 172.21.15.105:0/657791285 >> 172.21.15.105:0/657791285 conn(0x7fa2ac069c90 msgr2=0x7fa2ac06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.114+0000 7fa2b2e00640 1 -- 172.21.15.105:0/657791285 shutdown_connections 2024-03-23T07:56:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.114+0000 7fa2b2e00640 1 -- 172.21.15.105:0/657791285 wait complete. 2024-03-23T07:56:51.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.117+0000 7fa2b2e00640 1 Processor -- start 2024-03-23T07:56:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.117+0000 7fa2b2e00640 1 -- start start 2024-03-23T07:56:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.119+0000 7fa2b2e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2ac06a780 0x7fa2ac14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.119+0000 7fa2b2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac070170 0x7fa2ac0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.119+0000 7fa2b2e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac15c4f0 0x7fa2ac0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.119+0000 7fa2b2e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0b2f10 con 0x7fa2ac15c4f0 2024-03-23T07:56:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.119+0000 7fa2b2e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac153d10 con 0x7fa2ac06a780 2024-03-23T07:56:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.119+0000 7fa2b2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac153100 con 0x7fa2ac070170 2024-03-23T07:56:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.120+0000 7fa2abe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2ac06a780 0x7fa2ac14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.120+0000 7fa2abe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2ac06a780 0x7fa2ac14e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38278/0 (socket says 172.21.15.105:38278) 2024-03-23T07:56:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.120+0000 7fa2abe00640 1 -- 172.21.15.105:0/3231993068 learned_addr learned my addr 172.21.15.105:0/3231993068 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.120+0000 7fa2ab400640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac070170 0x7fa2ac0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.121+0000 7fa2b0e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac15c4f0 0x7fa2ac0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.122+0000 7fa2abe00640 1 -- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac070170 msgr2=0x7fa2ac0af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.122+0000 7fa2abe00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac070170 0x7fa2ac0af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.122+0000 7fa2abe00640 1 -- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac15c4f0 msgr2=0x7fa2ac0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.122+0000 7fa2abe00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac15c4f0 0x7fa2ac0aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.122+0000 7fa2abe00640 1 -- 172.21.15.105:0/3231993068 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa298009400 con 0x7fa2ac06a780 2024-03-23T07:56:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.122+0000 7fa2b0e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac15c4f0 0x7fa2ac0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:56:51.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.123+0000 7fa2abe00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2ac06a780 0x7fa2ac14e5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c780 tx=0x7fa29c00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.123+0000 7fa2a3e00640 1 -- 172.21.15.105:0/3231993068 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c005ab0 con 0x7fa2ac06a780 2024-03-23T07:56:51.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.124+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac159e20 con 0x7fa2ac06a780 2024-03-23T07:56:51.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.124+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac1536c0 con 0x7fa2ac06a780 2024-03-23T07:56:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.124+0000 7fa2a3e00640 1 -- 172.21.15.105:0/3231993068 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c005c10 con 0x7fa2ac06a780 2024-03-23T07:56:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.124+0000 7fa2a3e00640 1 -- 172.21.15.105:0/3231993068 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c007070 con 0x7fa2ac06a780 2024-03-23T07:56:51.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.126+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa284005250 con 0x7fa2ac06a780 2024-03-23T07:56:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.129+0000 7fa2a3e00640 1 -- 172.21.15.105:0/3231993068 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa29c0071d0 con 0x7fa2ac06a780 2024-03-23T07:56:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.130+0000 7fa2a3e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa28c06ac30 0x7fa28c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.130+0000 7fa2a3e00640 1 -- 172.21.15.105:0/3231993068 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa29c088be0 con 0x7fa2ac06a780 2024-03-23T07:56:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.131+0000 7fa2ab400640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa28c06ac30 0x7fa28c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.133+0000 7fa2ab400640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa28c06ac30 0x7fa28c06d0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa298034000 tx=0x7fa298008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.143+0000 7fa2a3e00640 1 -- 172.21.15.105:0/3231993068 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa29c053b70 con 0x7fa2ac06a780 2024-03-23T07:56:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.152+0000 7f664d200640 1 Processor -- start 2024-03-23T07:56:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.152+0000 7f664d200640 1 -- start start 2024-03-23T07:56:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.154+0000 7f664d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd1a0 0x7f6648164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.154+0000 7f664d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6648149be0 0x7f664814a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.154+0000 7f664d200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664806b200 con 0x7f66480cd580 2024-03-23T07:56:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.154+0000 7f664d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6648161340 con 0x7f6648149be0 2024-03-23T07:56:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.154+0000 7f664d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6648165920 con 0x7f66480cd1a0 2024-03-23T07:56:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.155+0000 7f6645400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f66480cd580 0x7f66481650f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59542/0 (socket says 172.21.15.105:59542) 2024-03-23T07:56:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.155+0000 7f6645400640 1 -- 172.21.15.105:0/3530033432 learned_addr learned my addr 172.21.15.105:0/3530033432 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.155+0000 7f6645e00640 1 --2- 172.21.15.105:0/3530033432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd1a0 0x7f6648164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.156+0000 7f6646800640 1 --2- 172.21.15.105:0/3530033432 >> v2:172.21.15.132:3300/0 conn(0x7f6648149be0 0x7f664814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.156+0000 7f6645e00640 1 -- 172.21.15.105:0/3530033432 >> v2:172.21.15.132:3300/0 conn(0x7f6648149be0 msgr2=0x7f664814a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.156+0000 7f6645e00640 1 --2- 172.21.15.105:0/3530033432 >> v2:172.21.15.132:3300/0 conn(0x7f6648149be0 0x7f664814a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.156+0000 7f6645e00640 1 -- 172.21.15.105:0/3530033432 >> v1:172.21.15.105:6789/0 conn(0x7f66480cd580 legacy=0x7f66481650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.156+0000 7f6645e00640 1 -- 172.21.15.105:0/3530033432 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664806b4e0 con 0x7f66480cd1a0 2024-03-23T07:56:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.158+0000 7f6645e00640 1 --2- 172.21.15.105:0/3530033432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd1a0 0x7f6648164bb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f66300097a0 tx=0x7f663002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f0e7d4c6450cc4f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.159+0000 7f6644a00640 1 -- 172.21.15.105:0/3530033432 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630031780 con 0x7f66480cd1a0 2024-03-23T07:56:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.159+0000 7f6644a00640 1 -- 172.21.15.105:0/3530033432 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66300318e0 con 0x7f66480cd1a0 2024-03-23T07:56:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.159+0000 7f6644a00640 1 -- 172.21.15.105:0/3530033432 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630031780 con 0x7f66480cd1a0 2024-03-23T07:56:51.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.160+0000 7f664d200640 1 -- 172.21.15.105:0/3530033432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd1a0 msgr2=0x7f6648164bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.160+0000 7f664d200640 1 --2- 172.21.15.105:0/3530033432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd1a0 0x7f6648164bb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f66300097a0 tx=0x7f663002b820 comp rx=0 tx=0).stop 2024-03-23T07:56:51.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.160+0000 7f664d200640 1 -- 172.21.15.105:0/3530033432 shutdown_connections 2024-03-23T07:56:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.160+0000 7f664d200640 1 --2- 172.21.15.105:0/3530033432 >> v2:172.21.15.132:3300/0 conn(0x7f6648149be0 0x7f664814a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.160+0000 7f664d200640 1 --2- 172.21.15.105:0/3530033432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd1a0 0x7f6648164bb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.160+0000 7f664d200640 1 -- 172.21.15.105:0/3530033432 >> 172.21.15.105:0/3530033432 conn(0x7f66480cf780 msgr2=0x7f66480cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.161+0000 7f664d200640 1 -- 172.21.15.105:0/3530033432 shutdown_connections 2024-03-23T07:56:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.161+0000 7f664d200640 1 -- 172.21.15.105:0/3530033432 wait complete. 2024-03-23T07:56:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.163+0000 7f664d200640 1 Processor -- start 2024-03-23T07:56:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.164+0000 7f664d200640 1 -- start start 2024-03-23T07:56:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.165+0000 7f664d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66480cd1a0 0x7f66480b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.166+0000 7f664d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd580 0x7f6648148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.166+0000 7f664d200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648149be0 0x7f664815f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.166+0000 7f6645e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66480cd1a0 0x7f66480b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.166+0000 7f6645e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66480cd1a0 0x7f66480b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38296/0 (socket says 172.21.15.105:38296) 2024-03-23T07:56:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.167+0000 7f6645e00640 1 -- 172.21.15.105:0/3031214163 learned_addr learned my addr 172.21.15.105:0/3031214163 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.167+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648161ce0 con 0x7f6648149be0 2024-03-23T07:56:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.167+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66480cc890 con 0x7f66480cd1a0 2024-03-23T07:56:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.167+0000 7f6646800640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648149be0 0x7f664815f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.167+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66480cc5b0 con 0x7f66480cd580 2024-03-23T07:56:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.168+0000 7f6645400640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd580 0x7f6648148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.169+0000 7f6645e00640 1 -- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd580 msgr2=0x7f6648148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.169+0000 7f6645e00640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd580 0x7f6648148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.169+0000 7f6645e00640 1 -- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648149be0 msgr2=0x7f664815f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.169+0000 7f6645e00640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648149be0 0x7f664815f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.169+0000 7f6645e00640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630009400 con 0x7f66480cd1a0 2024-03-23T07:56:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.169+0000 7f6646800640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648149be0 0x7f664815f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.170+0000 7f6645e00640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66480cd1a0 0x7f66480b7110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6630034040 tx=0x7f66300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.175+0000 7f663e000640 1 -- 172.21.15.105:0/3031214163 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630031780 con 0x7f66480cd1a0 2024-03-23T07:56:51.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.175+0000 7f663e000640 1 -- 172.21.15.105:0/3031214163 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630032a30 con 0x7f66480cd1a0 2024-03-23T07:56:51.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.175+0000 7f663e000640 1 -- 172.21.15.105:0/3031214163 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630032de0 con 0x7f66480cd1a0 2024-03-23T07:56:51.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.178+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6648161620 con 0x7f66480cd1a0 2024-03-23T07:56:51.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.178+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6648148770 con 0x7f66480cd1a0 2024-03-23T07:56:51.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.179+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6648148960 con 0x7f66480cd1a0 2024-03-23T07:56:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.182+0000 7f663e000640 1 -- 172.21.15.105:0/3031214163 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6630005990 con 0x7f66480cd1a0 2024-03-23T07:56:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.184+0000 7f663e000640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f662406acd0 0x7f662406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.184+0000 7f663e000640 1 -- 172.21.15.105:0/3031214163 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66300ab680 con 0x7f66480cd1a0 2024-03-23T07:56:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.185+0000 7f6645400640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f662406acd0 0x7f662406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.187+0000 7f6645400640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f662406acd0 0x7f662406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6648153c10 tx=0x7f6638009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.194+0000 7f663e000640 1 -- 172.21.15.105:0/3031214163 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6630076600 con 0x7f66480cd1a0 2024-03-23T07:56:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.389+0000 7f43e3200640 1 -- 172.21.15.105:0/2238606813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f43dc0b6640 con 0x7f43dc0b5b70 2024-03-23T07:56:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.390+0000 7f43cfe00640 1 -- 172.21.15.105:0/2238606813 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f43d407a820 con 0x7f43dc0b5b70 2024-03-23T07:56:51.394 INFO:teuthology.orchestra.run.smithi105.stdout:60129542154 2024-03-23T07:56:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.394+0000 7f43cd600640 1 -- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f43bc06acd0 msgr2=0x7f43bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.394+0000 7f43cd600640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f43bc06acd0 0x7f43bc06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0c9210 tx=0x7f43c8006f90 comp rx=0 tx=0).stop 2024-03-23T07:56:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.394+0000 7f43cd600640 1 -- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43dc0b5b70 msgr2=0x7f43dc15e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.394+0000 7f43cd600640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43dc0b5b70 0x7f43dc15e070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f43d4032000 tx=0x7f43d4004a70 comp rx=0 tx=0).stop 2024-03-23T07:56:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.395+0000 7f43cd600640 1 -- 172.21.15.105:0/2238606813 shutdown_connections 2024-03-23T07:56:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.395+0000 7f43cd600640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f43bc06acd0 0x7f43bc06d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.395+0000 7f43cd600640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43dc15c7b0 0x7f43dc15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.395+0000 7f43cd600640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43dc0b5b70 0x7f43dc15e070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.395+0000 7f43cd600640 1 --2- 172.21.15.105:0/2238606813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43dc0b5730 0x7f43dc15f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.395+0000 7f43cd600640 1 -- 172.21.15.105:0/2238606813 >> 172.21.15.105:0/2238606813 conn(0x7f43dc0717e0 msgr2=0x7f43dc071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.396+0000 7f43cd600640 1 -- 172.21.15.105:0/2238606813 shutdown_connections 2024-03-23T07:56:51.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.396+0000 7f43cd600640 1 -- 172.21.15.105:0/2238606813 wait complete. 2024-03-23T07:56:51.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.410+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbf701613d0 con 0x7fbf700cd830 2024-03-23T07:56:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.411+0000 7fbf66a00640 1 -- 172.21.15.105:0/2943074923 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf701613d0 con 0x7fbf700cd830 2024-03-23T07:56:51.414 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-03-23T07:56:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3838153e00 con 0x7f383815cd90 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf5006ac30 msgr2=0x7fbf5006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf5006ac30 0x7fbf5006d0f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf700b4930 tx=0x7fbf5c009040 comp rx=0 tx=0).stop 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 msgr2=0x7fbf700c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf700c7700 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c450 tx=0x7fbf6000c910 comp rx=0 tx=0).stop 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 shutdown_connections 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf7015c820 0x7fbf700b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf5006ac30 0x7fbf5006d0f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf700cf7c0 0x7fbf70151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 --2- 172.21.15.105:0/2943074923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf700cd830 0x7fbf700c7700 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c450 tx=0x7fbf6000c910 comp rx=0 tx=0).stop 2024-03-23T07:56:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.415+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 >> 172.21.15.105:0/2943074923 conn(0x7fbf7006b070 msgr2=0x7fbf700cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.416+0000 7f382e000640 1 -- 172.21.15.105:0/2392026681 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3838153e00 con 0x7f383815cd90 2024-03-23T07:56:51.419 INFO:teuthology.orchestra.run.smithi105.stdout:68719476747 2024-03-23T07:56:51.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.416+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 shutdown_connections 2024-03-23T07:56:51.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.416+0000 7fbf6fe00640 1 -- 172.21.15.105:0/2943074923 wait complete. 2024-03-23T07:56:51.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.419+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f381806acd0 msgr2=0x7f381806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.420+0000 7f383d200640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f381806acd0 0x7f381806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38380b3c90 tx=0x7f3828009110 comp rx=0 tx=0).stop 2024-03-23T07:56:51.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.420+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f383815cd90 msgr2=0x7f3838154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.420+0000 7f383d200640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f383815cd90 0x7f3838154f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3830032000 tx=0x7f3830004a40 comp rx=0 tx=0).stop 2024-03-23T07:56:51.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.420+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 shutdown_connections 2024-03-23T07:56:51.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.420+0000 7f383d200640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f383815cd90 0x7f3838154f10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.421+0000 7f383d200640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38380d2380 0x7f3838152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.421+0000 7f383d200640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f381806acd0 0x7f381806d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.421+0000 7f383d200640 1 --2- 172.21.15.105:0/2392026681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38380cd6b0 0x7f3838151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.421+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 >> 172.21.15.105:0/2392026681 conn(0x7f38380cf780 msgr2=0x7f38380d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.421+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 shutdown_connections 2024-03-23T07:56:51.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.422+0000 7f383d200640 1 -- 172.21.15.105:0/2392026681 wait complete. 2024-03-23T07:56:51.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.5 2024-03-23T07:56:51.425 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.439+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1d1415d050 con 0x7f1d140b1f20 2024-03-23T07:56:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.440+0000 7f1d0a000640 1 -- 172.21.15.105:0/914428654 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1cfc01e020 con 0x7f1d140b1f20 2024-03-23T07:56:51.441 INFO:teuthology.orchestra.run.smithi105.stdout:68719476749 2024-03-23T07:56:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.443+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cf006acd0 msgr2=0x7f1cf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.443+0000 7f1d18a00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cf006acd0 0x7f1cf006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d14071390 tx=0x7f1d0400d040 comp rx=0 tx=0).stop 2024-03-23T07:56:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d140b1f20 msgr2=0x7f1d1415fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d140b1f20 0x7f1d1415fba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c820 tx=0x7f1cfc00cce0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 shutdown_connections 2024-03-23T07:56:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cf006acd0 0x7f1cf006d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d1415cc70 0x7f1d140b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d140b4c90 0x7f1d140b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 --2- 172.21.15.105:0/914428654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d140b1f20 0x7f1d1415fba0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 >> 172.21.15.105:0/914428654 conn(0x7f1d140cf790 msgr2=0x7f1d140b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.7 2024-03-23T07:56:51.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476747 for osd.1 2024-03-23T07:56:51.447 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.447 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 shutdown_connections 2024-03-23T07:56:51.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.444+0000 7f1d18a00640 1 -- 172.21.15.105:0/914428654 wait complete. 2024-03-23T07:56:51.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476749 got 68719476749 for osd.0 2024-03-23T07:56:51.469 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.477+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9afc0089a0 con 0x7f9ae80047c0 2024-03-23T07:56:51.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.478+0000 7f9afa000640 1 -- 172.21.15.105:0/696181165 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9af4058a00 con 0x7f9ae80047c0 2024-03-23T07:56:51.478 INFO:teuthology.orchestra.run.smithi105.stdout:68719476747 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ae406ac00 msgr2=0x7f9ae406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ae406ac00 0x7f9ae406d0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9af0036000 tx=0x7f9af0031000 comp rx=0 tx=0).stop 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80047c0 msgr2=0x7f9afc04e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80047c0 0x7f9afc04e4b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9af4001640 tx=0x7f9af400d640 comp rx=0 tx=0).stop 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 shutdown_connections 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0d2310 0x7f9afc04c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc0d1e90 0x7f9afc04bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ae406ac00 0x7f9ae406d0c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 --2- 172.21.15.105:0/696181165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ae80047c0 0x7f9afc04e4b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 >> 172.21.15.105:0/696181165 conn(0x7f9afc009f80 msgr2=0x7f9afc00a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 shutdown_connections 2024-03-23T07:56:51.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.481+0000 7f9b03200640 1 -- 172.21.15.105:0/696181165 wait complete. 2024-03-23T07:56:51.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476747 for osd.3 2024-03-23T07:56:51.505 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.506+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f735c0aea80 con 0x7f73400047c0 2024-03-23T07:56:51.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.506+0000 7f7353e00640 1 -- 172.21.15.105:0/2060773119 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7354059a10 con 0x7f73400047c0 2024-03-23T07:56:51.507 INFO:teuthology.orchestra.run.smithi105.stdout:68719476747 2024-03-23T07:56:51.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.509+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f733c06ac00 msgr2=0x7f733c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.509+0000 7f7362a00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f733c06ac00 0x7f733c06d0c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7348033000 tx=0x7f7348009030 comp rx=0 tx=0).stop 2024-03-23T07:56:51.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73400047c0 msgr2=0x7f735c14d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73400047c0 0x7f735c14d2a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7354007fd0 tx=0x7f73540085a0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 shutdown_connections 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f733c06ac00 0x7f733c06d0c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b2840 0x7f735c1f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c156130 0x7f735c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 --2- 172.21.15.105:0/2060773119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73400047c0 0x7f735c14d2a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 >> 172.21.15.105:0/2060773119 conn(0x7f735c069c90 msgr2=0x7f735c070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 shutdown_connections 2024-03-23T07:56:51.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.510+0000 7f7362a00640 1 -- 172.21.15.105:0/2060773119 wait complete. 2024-03-23T07:56:51.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.514+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa284005500 con 0x7fa2ac06a780 2024-03-23T07:56:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.515+0000 7fa2a3e00640 1 -- 172.21.15.105:0/3231993068 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa29c017090 con 0x7fa2ac06a780 2024-03-23T07:56:51.516 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa28c06ac30 msgr2=0x7fa28c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa28c06ac30 0x7fa28c06d0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa298034000 tx=0x7fa298008030 comp rx=0 tx=0).stop 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2ac06a780 msgr2=0x7fa2ac14e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2ac06a780 0x7fa2ac14e5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c780 tx=0x7fa29c00cc40 comp rx=0 tx=0).stop 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 shutdown_connections 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ac15c4f0 0x7fa2ac0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa28c06ac30 0x7fa28c06d0f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2ac070170 0x7fa2ac0af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 --2- 172.21.15.105:0/3231993068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2ac06a780 0x7fa2ac14e5c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 >> 172.21.15.105:0/3231993068 conn(0x7fa2ac069c90 msgr2=0x7fa2ac156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 shutdown_connections 2024-03-23T07:56:51.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.517+0000 7fa2b2e00640 1 -- 172.21.15.105:0/3231993068 wait complete. 2024-03-23T07:56:51.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476747 for osd.2 2024-03-23T07:56:51.534 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.6 2024-03-23T07:56:51.535 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.563+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6648161480 con 0x7f66480cd1a0 2024-03-23T07:56:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.564+0000 7f663e000640 1 -- 172.21.15.105:0/3031214163 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6648161480 con 0x7f66480cd1a0 2024-03-23T07:56:51.565 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-03-23T07:56:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f662406acd0 msgr2=0x7f662406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f662406acd0 0x7f662406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6648153c10 tx=0x7f6638009040 comp rx=0 tx=0).stop 2024-03-23T07:56:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66480cd1a0 msgr2=0x7f66480b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66480cd1a0 0x7f66480b7110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6630034040 tx=0x7f66300046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 shutdown_connections 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648149be0 0x7f664815f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f662406acd0 0x7f662406d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66480cd580 0x7f6648148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.566+0000 7f664d200640 1 --2- 172.21.15.105:0/3031214163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66480cd1a0 0x7f66480b7110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.567+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 >> 172.21.15.105:0/3031214163 conn(0x7f66480cf780 msgr2=0x7f66480d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.567+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 shutdown_connections 2024-03-23T07:56:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.567+0000 7f664d200640 1 -- 172.21.15.105:0/3031214163 wait complete. 2024-03-23T07:56:51.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.4 2024-03-23T07:56:51.583 DEBUG:teuthology.parallel:result is None 2024-03-23T07:56:51.583 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-23T07:56:51.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:56:51.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.828+0000 7f0f57200640 1 Processor -- start 2024-03-23T07:56:51.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.828+0000 7f0f57200640 1 -- start start 2024-03-23T07:56:51.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.829+0000 7f0f57200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 0x7f0f580b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.829+0000 7f0f57200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f581575b0 con 0x7f0f580cdc00 2024-03-23T07:56:51.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.829+0000 7f0f57200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f58159f20 con 0x7f0f580cae50 2024-03-23T07:56:51.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.829+0000 7f0f57200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815a200 con 0x7f0f581571d0 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.830+0000 7f0f4d400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0f580cdc00 0x7f0f5815aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59546/0 (socket says 172.21.15.105:59546) 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.830+0000 7f0f4d400640 1 -- 172.21.15.105:0/4246195585 learned_addr learned my addr 172.21.15.105:0/4246195585 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.830+0000 7f0f55e00640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 0x7f0f580b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.830+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648843328 0 0) 0x7f0f581575b0 con 0x7f0f580cdc00 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.830+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f3c003520 con 0x7f0f580cdc00 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1426145125 0 0) 0x7f0f3c003520 con 0x7f0f580cdc00 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 msgr2=0x7f0f580b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 0x7f0f580b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 >> v1:172.21.15.132:6789/0 conn(0x7f0f580cae50 legacy=0x7f0f5815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5815a4e0 con 0x7f0f580cdc00 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f0f40001670 con 0x7f0f580cdc00 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 >> v1:172.21.15.105:6789/0 conn(0x7f0f580cdc00 legacy=0x7f0f5815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c0039a0 0x7f0f3c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f3c0047c0 0x7f0f3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 0x7f0f3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815a200 con 0x7f0f3c0039a0 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c00ba30 con 0x7f0f3c0047c0 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c00de00 con 0x7f0f581571d0 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55e00640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c0039a0 0x7f0f3c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f4d400640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f3c0047c0 0x7f0f3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f56800640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 0x7f0f3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55e00640 1 -- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 msgr2=0x7f0f3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55e00640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 0x7f0f3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55e00640 1 -- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f3c0047c0 msgr2=0x7f0f3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:51.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55e00640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f3c0047c0 0x7f0f3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f55e00640 1 -- 172.21.15.105:0/4246195585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c003520 con 0x7f0f3c0039a0 2024-03-23T07:56:51.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f56800640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f581571d0 0x7f0f3c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:51.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.831+0000 7f0f4d400640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f3c0047c0 0x7f0f3c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f55e00640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c0039a0 0x7f0f3c003dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009a90 tx=0x7f0f3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d316301f614ca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4802c680 con 0x7f0f3c0039a0 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4802c7e0 con 0x7f0f3c0039a0 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f55400640 1 -- 172.21.15.105:0/4246195585 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4802cb70 con 0x7f0f3c0039a0 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f57200640 1 -- 172.21.15.105:0/4246195585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c0039a0 msgr2=0x7f0f3c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f57200640 1 --2- 172.21.15.105:0/4246195585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f3c0039a0 0x7f0f3c003dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009a90 tx=0x7f0f3c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f56800640 1 -- 172.21.15.105:0/4246195585 reap_dead start 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f57200640 1 -- 172.21.15.105:0/4246195585 shutdown_connections 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f57200640 1 -- 172.21.15.105:0/4246195585 >> 172.21.15.105:0/4246195585 conn(0x7f0f58065df0 msgr2=0x7f0f58069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.833+0000 7f0f57200640 1 -- 172.21.15.105:0/4246195585 shutdown_connections 2024-03-23T07:56:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.834+0000 7f0f57200640 1 -- 172.21.15.105:0/4246195585 wait complete. 2024-03-23T07:56:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.835+0000 7f0f57200640 1 Processor -- start 2024-03-23T07:56:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.835+0000 7f0f57200640 1 -- start start 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.836+0000 7f0f57200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f3c0039a0 0x7f0f581ee290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.836+0000 7f0f57200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f58159430 0x7f0f581562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.836+0000 7f0f57200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580adfe0 0x7f0f58157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.836+0000 7f0f57200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581577e0 con 0x7f0f58159430 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.836+0000 7f0f57200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581ee7d0 con 0x7f0f580adfe0 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.836+0000 7f0f57200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581ef260 con 0x7f0f3c0039a0 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.836+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f58159430 0x7f0f581562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f58159430 0x7f0f581562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33724/0 (socket says 172.21.15.105:33724) 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 -- 172.21.15.105:0/133595016 learned_addr learned my addr 172.21.15.105:0/133595016 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f55e00640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f3c0039a0 0x7f0f581ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 -- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f3c0039a0 msgr2=0x7f0f581ee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f56800640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580adfe0 0x7f0f58157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f3c0039a0 0x7f0f581ee290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 -- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580adfe0 msgr2=0x7f0f58157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580adfe0 0x7f0f58157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 -- 172.21.15.105:0/133595016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f581ee9c0 con 0x7f0f58159430 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f55e00640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f3c0039a0 0x7f0f581ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4d400640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f58159430 0x7f0f581562e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000d810 tx=0x7f0f4000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4ea00640 1 -- 172.21.15.105:0/133595016 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400074e0 con 0x7f0f58159430 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4ea00640 1 -- 172.21.15.105:0/133595016 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40007640 con 0x7f0f58159430 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f4ea00640 1 -- 172.21.15.105:0/133595016 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40004a00 con 0x7f0f58159430 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.837+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581eeca0 con 0x7f0f58159430 2024-03-23T07:56:51.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.838+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f58156820 con 0x7f0f58159430 2024-03-23T07:56:51.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.839+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5804e030 con 0x7f0f58159430 2024-03-23T07:56:51.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.839+0000 7f0f4ea00640 1 -- 172.21.15.105:0/133595016 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0f40004b60 con 0x7f0f58159430 2024-03-23T07:56:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.840+0000 7f0f4ea00640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f3406ac00 0x7f0f3406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.840+0000 7f0f4ea00640 1 -- 172.21.15.105:0/133595016 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0f40088fa0 con 0x7f0f58159430 2024-03-23T07:56:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.840+0000 7f0f55e00640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f3406ac00 0x7f0f3406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.841+0000 7f0f55e00640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f3406ac00 0x7f0f3406d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f48036000 tx=0x7f0f48030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:51.845+0000 7f0f4ea00640 1 -- 172.21.15.105:0/133595016 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0f40053e80 con 0x7f0f58159430 2024-03-23T07:56:52.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.021+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f580b35d0 con 0x7f0f3406ac00 2024-03-23T07:56:52.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.035+0000 7f0f4ea00640 1 -- 172.21.15.105:0/133595016 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345042 (secure 0 0 0) 0x7f0f48004140 con 0x7f0f3406ac00 2024-03-23T07:56:52.039 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:52.041 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f3406ac00 msgr2=0x7f0f3406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f3406ac00 0x7f0f3406d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f48036000 tx=0x7f0f48030000 comp rx=0 tx=0).stop 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f58159430 msgr2=0x7f0f581562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f58159430 0x7f0f581562e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000d810 tx=0x7f0f4000dcd0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 shutdown_connections 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580adfe0 0x7f0f58157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f58159430 0x7f0f581562e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f3406ac00 0x7f0f3406d0c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 --2- 172.21.15.105:0/133595016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f3c0039a0 0x7f0f581ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 >> 172.21.15.105:0/133595016 conn(0x7f0f58065df0 msgr2=0x7f0f580b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:52.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 shutdown_connections 2024-03-23T07:56:52.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.040+0000 7f0f57200640 1 -- 172.21.15.105:0/133595016 wait complete. 2024-03-23T07:56:52.057 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-03-23T07:56:50.051796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2022002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004914"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995336+0000","last_change":"2024-03-23T07:56:17.227981+0000","last_active":"2024-03-23T07:56:21.995336+0000","last_peered":"2024-03-23T07:56:21.995336+0000","last_clean":"2024-03-23T07:56:21.995336+0000","last_became_active":"2024-03-23T07:56:17.227576+0000","last_became_peered":"2024-03-23T07:56:17.227576+0000","last_unstale":"2024-03-23T07:56:21.995336+0000","last_undegraded":"2024-03-23T07:56:21.995336+0000","last_fullsized":"2024-03-23T07:56:21.995336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001380+0000","last_change":"2024-03-23T07:56:17.730054+0000","last_active":"2024-03-23T07:56:22.001380+0000","last_peered":"2024-03-23T07:56:22.001380+0000","last_clean":"2024-03-23T07:56:22.001380+0000","last_became_active":"2024-03-23T07:56:17.729766+0000","last_became_peered":"2024-03-23T07:56:17.729766+0000","last_unstale":"2024-03-23T07:56:22.001380+0000","last_undegraded":"2024-03-23T07:56:22.001380+0000","last_fullsized":"2024-03-23T07:56:22.001380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:39.987411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.032954+0000","last_change":"2024-03-23T07:56:22.029375+0000","last_active":"2024-03-23T07:56:22.032954+0000","last_peered":"2024-03-23T07:56:22.032954+0000","last_clean":"2024-03-23T07:56:22.032954+0000","last_became_active":"2024-03-23T07:56:22.028383+0000","last_became_peered":"2024-03-23T07:56:22.028383+0000","last_unstale":"2024-03-23T07:56:22.032954+0000","last_undegraded":"2024-03-23T07:56:22.032954+0000","last_fullsized":"2024-03-23T07:56:22.032954+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:06.760193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001542+0000","last_change":"2024-03-23T07:56:17.217630+0000","last_active":"2024-03-23T07:56:22.001542+0000","last_peered":"2024-03-23T07:56:22.001542+0000","last_clean":"2024-03-23T07:56:22.001542+0000","last_became_active":"2024-03-23T07:56:17.217179+0000","last_became_peered":"2024-03-23T07:56:17.217179+0000","last_unstale":"2024-03-23T07:56:22.001542+0000","last_undegraded":"2024-03-23T07:56:22.001542+0000","last_fullsized":"2024-03-23T07:56:22.001542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:27:59.587867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004329+0000","last_change":"2024-03-23T07:56:17.734047+0000","last_active":"2024-03-23T07:56:22.004329+0000","last_peered":"2024-03-23T07:56:22.004329+0000","last_clean":"2024-03-23T07:56:22.004329+0000","last_became_active":"2024-03-23T07:56:17.733482+0000","last_became_peered":"2024-03-23T07:56:17.733482+0000","last_unstale":"2024-03-23T07:56:22.004329+0000","last_undegraded":"2024-03-23T07:56:22.004329+0000","last_fullsized":"2024-03-23T07:56:22.004329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:47:35.800634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093379+0000","last_change":"2024-03-23T07:56:22.012905+0000","last_active":"2024-03-23T07:56:22.093379+0000","last_peered":"2024-03-23T07:56:22.093379+0000","last_clean":"2024-03-23T07:56:22.093379+0000","last_became_active":"2024-03-23T07:56:22.012340+0000","last_became_peered":"2024-03-23T07:56:22.012340+0000","last_unstale":"2024-03-23T07:56:22.093379+0000","last_undegraded":"2024-03-23T07:56:22.093379+0000","last_fullsized":"2024-03-23T07:56:22.093379+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:43.862034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000002+0000","last_change":"2024-03-23T07:56:17.224614+0000","last_active":"2024-03-23T07:56:22.000002+0000","last_peered":"2024-03-23T07:56:22.000002+0000","last_clean":"2024-03-23T07:56:22.000002+0000","last_became_active":"2024-03-23T07:56:17.223463+0000","last_became_peered":"2024-03-23T07:56:17.223463+0000","last_unstale":"2024-03-23T07:56:22.000002+0000","last_undegraded":"2024-03-23T07:56:22.000002+0000","last_fullsized":"2024-03-23T07:56:22.000002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093607+0000","last_change":"2024-03-23T07:56:17.729551+0000","last_active":"2024-03-23T07:56:22.093607+0000","last_peered":"2024-03-23T07:56:22.093607+0000","last_clean":"2024-03-23T07:56:22.093607+0000","last_became_active":"2024-03-23T07:56:17.729295+0000","last_became_peered":"2024-03-23T07:56:17.729295+0000","last_unstale":"2024-03-23T07:56:22.093607+0000","last_undegraded":"2024-03-23T07:56:22.093607+0000","last_fullsized":"2024-03-23T07:56:22.093607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093533+0000","last_change":"2024-03-23T07:56:22.005574+0000","last_active":"2024-03-23T07:56:22.093533+0000","last_peered":"2024-03-23T07:56:22.093533+0000","last_clean":"2024-03-23T07:56:22.093533+0000","last_became_active":"2024-03-23T07:56:22.005169+0000","last_became_peered":"2024-03-23T07:56:22.005169+0000","last_unstale":"2024-03-23T07:56:22.093533+0000","last_undegraded":"2024-03-23T07:56:22.093533+0000","last_fullsized":"2024-03-23T07:56:22.093533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005504+0000","last_change":"2024-03-23T07:56:17.223235+0000","last_active":"2024-03-23T07:56:22.005504+0000","last_peered":"2024-03-23T07:56:22.005504+0000","last_clean":"2024-03-23T07:56:22.005504+0000","last_became_active":"2024-03-23T07:56:17.222925+0000","last_became_peered":"2024-03-23T07:56:17.222925+0000","last_unstale":"2024-03-23T07:56:22.005504+0000","last_undegraded":"2024-03-23T07:56:22.005504+0000","last_fullsized":"2024-03-23T07:56:22.005504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097254+0000","last_change":"2024-03-23T07:56:17.745979+0000","last_active":"2024-03-23T07:56:22.097254+0000","last_peered":"2024-03-23T07:56:22.097254+0000","last_clean":"2024-03-23T07:56:22.097254+0000","last_became_active":"2024-03-23T07:56:17.745702+0000","last_became_peered":"2024-03-23T07:56:17.745702+0000","last_unstale":"2024-03-23T07:56:22.097254+0000","last_undegraded":"2024-03-23T07:56:22.097254+0000","last_fullsized":"2024-03-23T07:56:22.097254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:07.760652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093472+0000","last_change":"2024-03-23T07:56:22.021092+0000","last_active":"2024-03-23T07:56:22.093472+0000","last_peered":"2024-03-23T07:56:22.093472+0000","last_clean":"2024-03-23T07:56:22.093472+0000","last_became_active":"2024-03-23T07:56:22.017901+0000","last_became_peered":"2024-03-23T07:56:22.017901+0000","last_unstale":"2024-03-23T07:56:22.093472+0000","last_undegraded":"2024-03-23T07:56:22.093472+0000","last_fullsized":"2024-03-23T07:56:22.093472+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097129+0000","last_change":"2024-03-23T07:56:17.733964+0000","last_active":"2024-03-23T07:56:22.097129+0000","last_peered":"2024-03-23T07:56:22.097129+0000","last_clean":"2024-03-23T07:56:22.097129+0000","last_became_active":"2024-03-23T07:56:17.733721+0000","last_became_peered":"2024-03-23T07:56:17.733721+0000","last_unstale":"2024-03-23T07:56:22.097129+0000","last_undegraded":"2024-03-23T07:56:22.097129+0000","last_fullsized":"2024-03-23T07:56:22.097129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004014+0000","last_change":"2024-03-23T07:56:17.227903+0000","last_active":"2024-03-23T07:56:22.004014+0000","last_peered":"2024-03-23T07:56:22.004014+0000","last_clean":"2024-03-23T07:56:22.004014+0000","last_became_active":"2024-03-23T07:56:17.227224+0000","last_became_peered":"2024-03-23T07:56:17.227224+0000","last_unstale":"2024-03-23T07:56:22.004014+0000","last_undegraded":"2024-03-23T07:56:22.004014+0000","last_fullsized":"2024-03-23T07:56:22.004014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097221+0000","last_change":"2024-03-23T07:56:22.023790+0000","last_active":"2024-03-23T07:56:22.097221+0000","last_peered":"2024-03-23T07:56:22.097221+0000","last_clean":"2024-03-23T07:56:22.097221+0000","last_became_active":"2024-03-23T07:56:22.022870+0000","last_became_peered":"2024-03-23T07:56:22.022870+0000","last_unstale":"2024-03-23T07:56:22.097221+0000","last_undegraded":"2024-03-23T07:56:22.097221+0000","last_fullsized":"2024-03-23T07:56:22.097221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:40.283526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014342+0000","last_change":"2024-03-23T07:56:17.734095+0000","last_active":"2024-03-23T07:56:22.014342+0000","last_peered":"2024-03-23T07:56:22.014342+0000","last_clean":"2024-03-23T07:56:22.014342+0000","last_became_active":"2024-03-23T07:56:17.733771+0000","last_became_peered":"2024-03-23T07:56:17.733771+0000","last_unstale":"2024-03-23T07:56:22.014342+0000","last_undegraded":"2024-03-23T07:56:22.014342+0000","last_fullsized":"2024-03-23T07:56:22.014342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002387+0000","last_change":"2024-03-23T07:56:17.221331+0000","last_active":"2024-03-23T07:56:22.002387+0000","last_peered":"2024-03-23T07:56:22.002387+0000","last_clean":"2024-03-23T07:56:22.002387+0000","last_became_active":"2024-03-23T07:56:17.221062+0000","last_became_peered":"2024-03-23T07:56:17.221062+0000","last_unstale":"2024-03-23T07:56:22.002387+0000","last_undegraded":"2024-03-23T07:56:22.002387+0000","last_fullsized":"2024-03-23T07:56:22.002387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096460+0000","last_change":"2024-03-23T07:56:22.028190+0000","last_active":"2024-03-23T07:56:22.096460+0000","last_peered":"2024-03-23T07:56:22.096460+0000","last_clean":"2024-03-23T07:56:22.096460+0000","last_became_active":"2024-03-23T07:56:22.027188+0000","last_became_peered":"2024-03-23T07:56:22.027188+0000","last_unstale":"2024-03-23T07:56:22.096460+0000","last_undegraded":"2024-03-23T07:56:22.096460+0000","last_fullsized":"2024-03-23T07:56:22.096460+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985338+0000","last_change":"2024-03-23T07:56:17.728823+0000","last_active":"2024-03-23T07:56:21.985338+0000","last_peered":"2024-03-23T07:56:21.985338+0000","last_clean":"2024-03-23T07:56:21.985338+0000","last_became_active":"2024-03-23T07:56:17.728504+0000","last_became_peered":"2024-03-23T07:56:17.728504+0000","last_unstale":"2024-03-23T07:56:21.985338+0000","last_undegraded":"2024-03-23T07:56:21.985338+0000","last_fullsized":"2024-03-23T07:56:21.985338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:28.882493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984446+0000","last_change":"2024-03-23T07:56:17.209430+0000","last_active":"2024-03-23T07:56:21.984446+0000","last_peered":"2024-03-23T07:56:21.984446+0000","last_clean":"2024-03-23T07:56:21.984446+0000","last_became_active":"2024-03-23T07:56:17.207873+0000","last_became_peered":"2024-03-23T07:56:17.207873+0000","last_unstale":"2024-03-23T07:56:21.984446+0000","last_undegraded":"2024-03-23T07:56:21.984446+0000","last_fullsized":"2024-03-23T07:56:21.984446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.033429+0000","last_change":"2024-03-23T07:56:22.031440+0000","last_active":"2024-03-23T07:56:22.033429+0000","last_peered":"2024-03-23T07:56:22.033429+0000","last_clean":"2024-03-23T07:56:22.033429+0000","last_became_active":"2024-03-23T07:56:22.030403+0000","last_became_peered":"2024-03-23T07:56:22.030403+0000","last_unstale":"2024-03-23T07:56:22.033429+0000","last_undegraded":"2024-03-23T07:56:22.033429+0000","last_fullsized":"2024-03-23T07:56:22.033429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:40.289743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097084+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097084+0000","last_peered":"2024-03-23T07:56:22.097084+0000","last_clean":"2024-03-23T07:56:22.097084+0000","last_became_active":"2024-03-23T07:56:17.728164+0000","last_became_peered":"2024-03-23T07:56:17.728164+0000","last_unstale":"2024-03-23T07:56:22.097084+0000","last_undegraded":"2024-03-23T07:56:22.097084+0000","last_fullsized":"2024-03-23T07:56:22.097084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:11:45.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998174+0000","last_change":"2024-03-23T07:56:17.211209+0000","last_active":"2024-03-23T07:56:21.998174+0000","last_peered":"2024-03-23T07:56:21.998174+0000","last_clean":"2024-03-23T07:56:21.998174+0000","last_became_active":"2024-03-23T07:56:17.210792+0000","last_became_peered":"2024-03-23T07:56:17.210792+0000","last_unstale":"2024-03-23T07:56:21.998174+0000","last_undegraded":"2024-03-23T07:56:21.998174+0000","last_fullsized":"2024-03-23T07:56:21.998174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093682+0000","last_change":"2024-03-23T07:56:22.016254+0000","last_active":"2024-03-23T07:56:22.093682+0000","last_peered":"2024-03-23T07:56:22.093682+0000","last_clean":"2024-03-23T07:56:22.093682+0000","last_became_active":"2024-03-23T07:56:22.015890+0000","last_became_peered":"2024-03-23T07:56:22.015890+0000","last_unstale":"2024-03-23T07:56:22.093682+0000","last_undegraded":"2024-03-23T07:56:22.093682+0000","last_fullsized":"2024-03-23T07:56:22.093682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:15.477490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992721+0000","last_change":"2024-03-23T07:56:17.726007+0000","last_active":"2024-03-23T07:56:21.992721+0000","last_peered":"2024-03-23T07:56:21.992721+0000","last_clean":"2024-03-23T07:56:21.992721+0000","last_became_active":"2024-03-23T07:56:17.723965+0000","last_became_peered":"2024-03-23T07:56:17.723965+0000","last_unstale":"2024-03-23T07:56:21.992721+0000","last_undegraded":"2024-03-23T07:56:21.992721+0000","last_fullsized":"2024-03-23T07:56:21.992721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004697+0000","last_change":"2024-03-23T07:56:17.225692+0000","last_active":"2024-03-23T07:56:22.004697+0000","last_peered":"2024-03-23T07:56:22.004697+0000","last_clean":"2024-03-23T07:56:22.004697+0000","last_became_active":"2024-03-23T07:56:17.225384+0000","last_became_peered":"2024-03-23T07:56:17.225384+0000","last_unstale":"2024-03-23T07:56:22.004697+0000","last_undegraded":"2024-03-23T07:56:22.004697+0000","last_fullsized":"2024-03-23T07:56:22.004697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:21.655121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093432+0000","last_change":"2024-03-23T07:56:22.007586+0000","last_active":"2024-03-23T07:56:22.093432+0000","last_peered":"2024-03-23T07:56:22.093432+0000","last_clean":"2024-03-23T07:56:22.093432+0000","last_became_active":"2024-03-23T07:56:22.007159+0000","last_became_peered":"2024-03-23T07:56:22.007159+0000","last_unstale":"2024-03-23T07:56:22.093432+0000","last_undegraded":"2024-03-23T07:56:22.093432+0000","last_fullsized":"2024-03-23T07:56:22.093432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995105+0000","last_change":"2024-03-23T07:56:17.722742+0000","last_active":"2024-03-23T07:56:21.995105+0000","last_peered":"2024-03-23T07:56:21.995105+0000","last_clean":"2024-03-23T07:56:21.995105+0000","last_became_active":"2024-03-23T07:56:17.722246+0000","last_became_peered":"2024-03-23T07:56:17.722246+0000","last_unstale":"2024-03-23T07:56:21.995105+0000","last_undegraded":"2024-03-23T07:56:21.995105+0000","last_fullsized":"2024-03-23T07:56:21.995105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994508+0000","last_change":"2024-03-23T07:56:17.212269+0000","last_active":"2024-03-23T07:56:21.994508+0000","last_peered":"2024-03-23T07:56:21.994508+0000","last_clean":"2024-03-23T07:56:21.994508+0000","last_became_active":"2024-03-23T07:56:17.211965+0000","last_became_peered":"2024-03-23T07:56:17.211965+0000","last_unstale":"2024-03-23T07:56:21.994508+0000","last_undegraded":"2024-03-23T07:56:21.994508+0000","last_fullsized":"2024-03-23T07:56:21.994508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015727+0000","last_change":"2024-03-23T07:56:22.009793+0000","last_active":"2024-03-23T07:56:22.015727+0000","last_peered":"2024-03-23T07:56:22.015727+0000","last_clean":"2024-03-23T07:56:22.015727+0000","last_became_active":"2024-03-23T07:56:22.009417+0000","last_became_peered":"2024-03-23T07:56:22.009417+0000","last_unstale":"2024-03-23T07:56:22.015727+0000","last_undegraded":"2024-03-23T07:56:22.015727+0000","last_fullsized":"2024-03-23T07:56:22.015727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003091+0000","last_change":"2024-03-23T07:56:17.726582+0000","last_active":"2024-03-23T07:56:22.003091+0000","last_peered":"2024-03-23T07:56:22.003091+0000","last_clean":"2024-03-23T07:56:22.003091+0000","last_became_active":"2024-03-23T07:56:17.726338+0000","last_became_peered":"2024-03-23T07:56:17.726338+0000","last_unstale":"2024-03-23T07:56:22.003091+0000","last_undegraded":"2024-03-23T07:56:22.003091+0000","last_fullsized":"2024-03-23T07:56:22.003091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004594+0000","last_change":"2024-03-23T07:56:17.215985+0000","last_active":"2024-03-23T07:56:22.004594+0000","last_peered":"2024-03-23T07:56:22.004594+0000","last_clean":"2024-03-23T07:56:22.004594+0000","last_became_active":"2024-03-23T07:56:17.215709+0000","last_became_peered":"2024-03-23T07:56:17.215709+0000","last_unstale":"2024-03-23T07:56:22.004594+0000","last_undegraded":"2024-03-23T07:56:22.004594+0000","last_fullsized":"2024-03-23T07:56:22.004594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028361+0000","last_change":"2024-03-23T07:56:22.026672+0000","last_active":"2024-03-23T07:56:22.028361+0000","last_peered":"2024-03-23T07:56:22.028361+0000","last_clean":"2024-03-23T07:56:22.028361+0000","last_became_active":"2024-03-23T07:56:22.025340+0000","last_became_peered":"2024-03-23T07:56:22.025340+0000","last_unstale":"2024-03-23T07:56:22.028361+0000","last_undegraded":"2024-03-23T07:56:22.028361+0000","last_fullsized":"2024-03-23T07:56:22.028361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:59.226182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093663+0000","last_change":"2024-03-23T07:56:17.723057+0000","last_active":"2024-03-23T07:56:22.093663+0000","last_peered":"2024-03-23T07:56:22.093663+0000","last_clean":"2024-03-23T07:56:22.093663+0000","last_became_active":"2024-03-23T07:56:17.722787+0000","last_became_peered":"2024-03-23T07:56:17.722787+0000","last_unstale":"2024-03-23T07:56:22.093663+0000","last_undegraded":"2024-03-23T07:56:22.093663+0000","last_fullsized":"2024-03-23T07:56:22.093663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992051+0000","last_change":"2024-03-23T07:56:17.217582+0000","last_active":"2024-03-23T07:56:21.992051+0000","last_peered":"2024-03-23T07:56:21.992051+0000","last_clean":"2024-03-23T07:56:21.992051+0000","last_became_active":"2024-03-23T07:56:17.217205+0000","last_became_peered":"2024-03-23T07:56:17.217205+0000","last_unstale":"2024-03-23T07:56:21.992051+0000","last_undegraded":"2024-03-23T07:56:21.992051+0000","last_fullsized":"2024-03-23T07:56:21.992051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:23:32.182551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.027442+0000","last_change":"2024-03-23T07:56:22.023281+0000","last_active":"2024-03-23T07:56:22.027442+0000","last_peered":"2024-03-23T07:56:22.027442+0000","last_clean":"2024-03-23T07:56:22.027442+0000","last_became_active":"2024-03-23T07:56:22.022756+0000","last_became_peered":"2024-03-23T07:56:22.022756+0000","last_unstale":"2024-03-23T07:56:22.027442+0000","last_undegraded":"2024-03-23T07:56:22.027442+0000","last_fullsized":"2024-03-23T07:56:22.027442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096425+0000","last_change":"2024-03-23T07:56:17.740008+0000","last_active":"2024-03-23T07:56:22.096425+0000","last_peered":"2024-03-23T07:56:22.096425+0000","last_clean":"2024-03-23T07:56:22.096425+0000","last_became_active":"2024-03-23T07:56:17.739785+0000","last_became_peered":"2024-03-23T07:56:17.739785+0000","last_unstale":"2024-03-23T07:56:22.096425+0000","last_undegraded":"2024-03-23T07:56:22.096425+0000","last_fullsized":"2024-03-23T07:56:22.096425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003892+0000","last_change":"2024-03-23T07:56:17.221620+0000","last_active":"2024-03-23T07:56:22.003892+0000","last_peered":"2024-03-23T07:56:22.003892+0000","last_clean":"2024-03-23T07:56:22.003892+0000","last_became_active":"2024-03-23T07:56:17.221312+0000","last_became_peered":"2024-03-23T07:56:17.221312+0000","last_unstale":"2024-03-23T07:56:22.003892+0000","last_undegraded":"2024-03-23T07:56:22.003892+0000","last_fullsized":"2024-03-23T07:56:22.003892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:47.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.024787+0000","last_change":"2024-03-23T07:56:22.018139+0000","last_active":"2024-03-23T07:56:22.024787+0000","last_peered":"2024-03-23T07:56:22.024787+0000","last_clean":"2024-03-23T07:56:22.024787+0000","last_became_active":"2024-03-23T07:56:22.017722+0000","last_became_peered":"2024-03-23T07:56:22.017722+0000","last_unstale":"2024-03-23T07:56:22.024787+0000","last_undegraded":"2024-03-23T07:56:22.024787+0000","last_fullsized":"2024-03-23T07:56:22.024787+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983276+0000","last_change":"2024-03-23T07:56:17.729485+0000","last_active":"2024-03-23T07:56:21.983276+0000","last_peered":"2024-03-23T07:56:21.983276+0000","last_clean":"2024-03-23T07:56:21.983276+0000","last_became_active":"2024-03-23T07:56:17.729199+0000","last_became_peered":"2024-03-23T07:56:17.729199+0000","last_unstale":"2024-03-23T07:56:21.983276+0000","last_undegraded":"2024-03-23T07:56:21.983276+0000","last_fullsized":"2024-03-23T07:56:21.983276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:30:59.262864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014955+0000","last_change":"2024-03-23T07:56:17.222373+0000","last_active":"2024-03-23T07:56:22.014955+0000","last_peered":"2024-03-23T07:56:22.014955+0000","last_clean":"2024-03-23T07:56:22.014955+0000","last_became_active":"2024-03-23T07:56:17.222028+0000","last_became_peered":"2024-03-23T07:56:17.222028+0000","last_unstale":"2024-03-23T07:56:22.014955+0000","last_undegraded":"2024-03-23T07:56:22.014955+0000","last_fullsized":"2024-03-23T07:56:22.014955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.023027+0000","last_change":"2024-03-23T07:56:22.018041+0000","last_active":"2024-03-23T07:56:22.023027+0000","last_peered":"2024-03-23T07:56:22.023027+0000","last_clean":"2024-03-23T07:56:22.023027+0000","last_became_active":"2024-03-23T07:56:22.017588+0000","last_became_peered":"2024-03-23T07:56:22.017588+0000","last_unstale":"2024-03-23T07:56:22.023027+0000","last_undegraded":"2024-03-23T07:56:22.023027+0000","last_fullsized":"2024-03-23T07:56:22.023027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001772+0000","last_change":"2024-03-23T07:56:17.724698+0000","last_active":"2024-03-23T07:56:22.001772+0000","last_peered":"2024-03-23T07:56:22.001772+0000","last_clean":"2024-03-23T07:56:22.001772+0000","last_became_active":"2024-03-23T07:56:17.724382+0000","last_became_peered":"2024-03-23T07:56:17.724382+0000","last_unstale":"2024-03-23T07:56:22.001772+0000","last_undegraded":"2024-03-23T07:56:22.001772+0000","last_fullsized":"2024-03-23T07:56:22.001772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002505+0000","last_change":"2024-03-23T07:56:17.223691+0000","last_active":"2024-03-23T07:56:22.002505+0000","last_peered":"2024-03-23T07:56:22.002505+0000","last_clean":"2024-03-23T07:56:22.002505+0000","last_became_active":"2024-03-23T07:56:17.222061+0000","last_became_peered":"2024-03-23T07:56:17.222061+0000","last_unstale":"2024-03-23T07:56:22.002505+0000","last_undegraded":"2024-03-23T07:56:22.002505+0000","last_fullsized":"2024-03-23T07:56:22.002505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:00.798128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471272+0000","last_change":"2024-03-23T07:56:22.011743+0000","last_active":"2024-03-23T07:56:22.471272+0000","last_peered":"2024-03-23T07:56:22.471272+0000","last_clean":"2024-03-23T07:56:22.471272+0000","last_became_active":"2024-03-23T07:56:22.011280+0000","last_became_peered":"2024-03-23T07:56:22.011280+0000","last_unstale":"2024-03-23T07:56:22.471272+0000","last_undegraded":"2024-03-23T07:56:22.471272+0000","last_fullsized":"2024-03-23T07:56:22.471272+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:05.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988647+0000","last_change":"2024-03-23T07:56:17.722136+0000","last_active":"2024-03-23T07:56:21.988647+0000","last_peered":"2024-03-23T07:56:21.988647+0000","last_clean":"2024-03-23T07:56:21.988647+0000","last_became_active":"2024-03-23T07:56:17.720479+0000","last_became_peered":"2024-03-23T07:56:17.720479+0000","last_unstale":"2024-03-23T07:56:21.988647+0000","last_undegraded":"2024-03-23T07:56:21.988647+0000","last_fullsized":"2024-03-23T07:56:21.988647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003825+0000","last_change":"2024-03-23T07:56:17.227266+0000","last_active":"2024-03-23T07:56:22.003825+0000","last_peered":"2024-03-23T07:56:22.003825+0000","last_clean":"2024-03-23T07:56:22.003825+0000","last_became_active":"2024-03-23T07:56:17.226775+0000","last_became_peered":"2024-03-23T07:56:17.226775+0000","last_unstale":"2024-03-23T07:56:22.003825+0000","last_undegraded":"2024-03-23T07:56:22.003825+0000","last_fullsized":"2024-03-23T07:56:22.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005759+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:56:22.005759+0000","last_peered":"2024-03-23T07:56:22.005759+0000","last_clean":"2024-03-23T07:56:22.005759+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:56:22.005759+0000","last_undegraded":"2024-03-23T07:56:22.005759+0000","last_fullsized":"2024-03-23T07:56:22.005759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097280+0000","last_change":"2024-03-23T07:56:22.024202+0000","last_active":"2024-03-23T07:56:22.097280+0000","last_peered":"2024-03-23T07:56:22.097280+0000","last_clean":"2024-03-23T07:56:22.097280+0000","last_became_active":"2024-03-23T07:56:22.018941+0000","last_became_peered":"2024-03-23T07:56:22.018941+0000","last_unstale":"2024-03-23T07:56:22.097280+0000","last_undegraded":"2024-03-23T07:56:22.097280+0000","last_fullsized":"2024-03-23T07:56:22.097280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:25.917706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097267+0000","last_change":"2024-03-23T07:56:17.734848+0000","last_active":"2024-03-23T07:56:22.097267+0000","last_peered":"2024-03-23T07:56:22.097267+0000","last_clean":"2024-03-23T07:56:22.097267+0000","last_became_active":"2024-03-23T07:56:17.733233+0000","last_became_peered":"2024-03-23T07:56:17.733233+0000","last_unstale":"2024-03-23T07:56:22.097267+0000","last_undegraded":"2024-03-23T07:56:22.097267+0000","last_fullsized":"2024-03-23T07:56:22.097267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003338+0000","last_change":"2024-03-23T07:56:17.215068+0000","last_active":"2024-03-23T07:56:22.003338+0000","last_peered":"2024-03-23T07:56:22.003338+0000","last_clean":"2024-03-23T07:56:22.003338+0000","last_became_active":"2024-03-23T07:56:17.214726+0000","last_became_peered":"2024-03-23T07:56:17.214726+0000","last_unstale":"2024-03-23T07:56:22.003338+0000","last_undegraded":"2024-03-23T07:56:22.003338+0000","last_fullsized":"2024-03-23T07:56:22.003338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093494+0000","last_change":"2024-03-23T07:56:22.023739+0000","last_active":"2024-03-23T07:56:22.093494+0000","last_peered":"2024-03-23T07:56:22.093494+0000","last_clean":"2024-03-23T07:56:22.093494+0000","last_became_active":"2024-03-23T07:56:22.023366+0000","last_became_peered":"2024-03-23T07:56:22.023366+0000","last_unstale":"2024-03-23T07:56:22.093494+0000","last_undegraded":"2024-03-23T07:56:22.093494+0000","last_fullsized":"2024-03-23T07:56:22.093494+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:13.143472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096511+0000","last_change":"2024-03-23T07:56:17.737947+0000","last_active":"2024-03-23T07:56:22.096511+0000","last_peered":"2024-03-23T07:56:22.096511+0000","last_clean":"2024-03-23T07:56:22.096511+0000","last_became_active":"2024-03-23T07:56:17.733346+0000","last_became_peered":"2024-03-23T07:56:17.733346+0000","last_unstale":"2024-03-23T07:56:22.096511+0000","last_undegraded":"2024-03-23T07:56:22.096511+0000","last_fullsized":"2024-03-23T07:56:22.096511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998748+0000","last_change":"2024-03-23T07:56:17.221283+0000","last_active":"2024-03-23T07:56:21.998748+0000","last_peered":"2024-03-23T07:56:21.998748+0000","last_clean":"2024-03-23T07:56:21.998748+0000","last_became_active":"2024-03-23T07:56:17.219961+0000","last_became_peered":"2024-03-23T07:56:17.219961+0000","last_unstale":"2024-03-23T07:56:21.998748+0000","last_undegraded":"2024-03-23T07:56:21.998748+0000","last_fullsized":"2024-03-23T07:56:21.998748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:04:11.860279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.470374+0000","last_change":"2024-03-23T07:56:22.016551+0000","last_active":"2024-03-23T07:56:22.470374+0000","last_peered":"2024-03-23T07:56:22.470374+0000","last_clean":"2024-03-23T07:56:22.470374+0000","last_became_active":"2024-03-23T07:56:22.016136+0000","last_became_peered":"2024-03-23T07:56:22.016136+0000","last_unstale":"2024-03-23T07:56:22.470374+0000","last_undegraded":"2024-03-23T07:56:22.470374+0000","last_fullsized":"2024-03-23T07:56:22.470374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:36.873288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096655+0000","last_change":"2024-03-23T07:56:17.743611+0000","last_active":"2024-03-23T07:56:22.096655+0000","last_peered":"2024-03-23T07:56:22.096655+0000","last_clean":"2024-03-23T07:56:22.096655+0000","last_became_active":"2024-03-23T07:56:17.742926+0000","last_became_peered":"2024-03-23T07:56:17.742926+0000","last_unstale":"2024-03-23T07:56:22.096655+0000","last_undegraded":"2024-03-23T07:56:22.096655+0000","last_fullsized":"2024-03-23T07:56:22.096655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994328+0000","last_change":"2024-03-23T07:56:17.219709+0000","last_active":"2024-03-23T07:56:21.994328+0000","last_peered":"2024-03-23T07:56:21.994328+0000","last_clean":"2024-03-23T07:56:21.994328+0000","last_became_active":"2024-03-23T07:56:17.218465+0000","last_became_peered":"2024-03-23T07:56:17.218465+0000","last_unstale":"2024-03-23T07:56:21.994328+0000","last_undegraded":"2024-03-23T07:56:21.994328+0000","last_fullsized":"2024-03-23T07:56:21.994328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096964+0000","last_change":"2024-03-23T07:56:22.017346+0000","last_active":"2024-03-23T07:56:22.096964+0000","last_peered":"2024-03-23T07:56:22.096964+0000","last_clean":"2024-03-23T07:56:22.096964+0000","last_became_active":"2024-03-23T07:56:22.016949+0000","last_became_peered":"2024-03-23T07:56:22.016949+0000","last_unstale":"2024-03-23T07:56:22.096964+0000","last_undegraded":"2024-03-23T07:56:22.096964+0000","last_fullsized":"2024-03-23T07:56:22.096964+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:45.299178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093733+0000","last_change":"2024-03-23T07:56:17.742996+0000","last_active":"2024-03-23T07:56:22.093733+0000","last_peered":"2024-03-23T07:56:22.093733+0000","last_clean":"2024-03-23T07:56:22.093733+0000","last_became_active":"2024-03-23T07:56:17.742660+0000","last_became_peered":"2024-03-23T07:56:17.742660+0000","last_unstale":"2024-03-23T07:56:22.093733+0000","last_undegraded":"2024-03-23T07:56:22.093733+0000","last_fullsized":"2024-03-23T07:56:22.093733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004353+0000","last_change":"2024-03-23T07:56:17.215901+0000","last_active":"2024-03-23T07:56:22.004353+0000","last_peered":"2024-03-23T07:56:22.004353+0000","last_clean":"2024-03-23T07:56:22.004353+0000","last_became_active":"2024-03-23T07:56:17.215638+0000","last_became_peered":"2024-03-23T07:56:17.215638+0000","last_unstale":"2024-03-23T07:56:22.004353+0000","last_undegraded":"2024-03-23T07:56:22.004353+0000","last_fullsized":"2024-03-23T07:56:22.004353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093529+0000","last_change":"2024-03-23T07:56:22.010512+0000","last_active":"2024-03-23T07:56:22.093529+0000","last_peered":"2024-03-23T07:56:22.093529+0000","last_clean":"2024-03-23T07:56:22.093529+0000","last_became_active":"2024-03-23T07:56:22.009577+0000","last_became_peered":"2024-03-23T07:56:22.009577+0000","last_unstale":"2024-03-23T07:56:22.093529+0000","last_undegraded":"2024-03-23T07:56:22.093529+0000","last_fullsized":"2024-03-23T07:56:22.093529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:24.844391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097072+0000","last_change":"2024-03-23T07:56:17.732277+0000","last_active":"2024-03-23T07:56:22.097072+0000","last_peered":"2024-03-23T07:56:22.097072+0000","last_clean":"2024-03-23T07:56:22.097072+0000","last_became_active":"2024-03-23T07:56:17.731487+0000","last_became_peered":"2024-03-23T07:56:17.731487+0000","last_unstale":"2024-03-23T07:56:22.097072+0000","last_undegraded":"2024-03-23T07:56:22.097072+0000","last_fullsized":"2024-03-23T07:56:22.097072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:39.462067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985538+0000","last_change":"2024-03-23T07:56:17.211010+0000","last_active":"2024-03-23T07:56:21.985538+0000","last_peered":"2024-03-23T07:56:21.985538+0000","last_clean":"2024-03-23T07:56:21.985538+0000","last_became_active":"2024-03-23T07:56:17.210710+0000","last_became_peered":"2024-03-23T07:56:17.210710+0000","last_unstale":"2024-03-23T07:56:21.985538+0000","last_undegraded":"2024-03-23T07:56:21.985538+0000","last_fullsized":"2024-03-23T07:56:21.985538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471596+0000","last_change":"2024-03-23T07:56:22.019325+0000","last_active":"2024-03-23T07:56:22.471596+0000","last_peered":"2024-03-23T07:56:22.471596+0000","last_clean":"2024-03-23T07:56:22.471596+0000","last_became_active":"2024-03-23T07:56:22.016793+0000","last_became_peered":"2024-03-23T07:56:22.016793+0000","last_unstale":"2024-03-23T07:56:22.471596+0000","last_undegraded":"2024-03-23T07:56:22.471596+0000","last_fullsized":"2024-03-23T07:56:22.471596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:08:59.171123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093481+0000","last_change":"2024-03-23T07:56:17.716144+0000","last_active":"2024-03-23T07:56:22.093481+0000","last_peered":"2024-03-23T07:56:22.093481+0000","last_clean":"2024-03-23T07:56:22.093481+0000","last_became_active":"2024-03-23T07:56:17.715899+0000","last_became_peered":"2024-03-23T07:56:17.715899+0000","last_unstale":"2024-03-23T07:56:22.093481+0000","last_undegraded":"2024-03-23T07:56:22.093481+0000","last_fullsized":"2024-03-23T07:56:22.093481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005380+0000","last_change":"2024-03-23T07:56:17.226828+0000","last_active":"2024-03-23T07:56:22.005380+0000","last_peered":"2024-03-23T07:56:22.005380+0000","last_clean":"2024-03-23T07:56:22.005380+0000","last_became_active":"2024-03-23T07:56:17.226551+0000","last_became_peered":"2024-03-23T07:56:17.226551+0000","last_unstale":"2024-03-23T07:56:22.005380+0000","last_undegraded":"2024-03-23T07:56:22.005380+0000","last_fullsized":"2024-03-23T07:56:22.005380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:03.843216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471755+0000","last_change":"2024-03-23T07:56:22.019606+0000","last_active":"2024-03-23T07:56:22.471755+0000","last_peered":"2024-03-23T07:56:22.471755+0000","last_clean":"2024-03-23T07:56:22.471755+0000","last_became_active":"2024-03-23T07:56:22.019280+0000","last_became_peered":"2024-03-23T07:56:22.019280+0000","last_unstale":"2024-03-23T07:56:22.471755+0000","last_undegraded":"2024-03-23T07:56:22.471755+0000","last_fullsized":"2024-03-23T07:56:22.471755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:28.706338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097162+0000","last_change":"2024-03-23T07:56:17.746537+0000","last_active":"2024-03-23T07:56:22.097162+0000","last_peered":"2024-03-23T07:56:22.097162+0000","last_clean":"2024-03-23T07:56:22.097162+0000","last_became_active":"2024-03-23T07:56:17.746054+0000","last_became_peered":"2024-03-23T07:56:17.746054+0000","last_unstale":"2024-03-23T07:56:22.097162+0000","last_undegraded":"2024-03-23T07:56:22.097162+0000","last_fullsized":"2024-03-23T07:56:22.097162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:09.358882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003661+0000","last_change":"2024-03-23T07:56:17.225527+0000","last_active":"2024-03-23T07:56:22.003661+0000","last_peered":"2024-03-23T07:56:22.003661+0000","last_clean":"2024-03-23T07:56:22.003661+0000","last_became_active":"2024-03-23T07:56:17.224860+0000","last_became_peered":"2024-03-23T07:56:17.224860+0000","last_unstale":"2024-03-23T07:56:22.003661+0000","last_undegraded":"2024-03-23T07:56:22.003661+0000","last_fullsized":"2024-03-23T07:56:22.003661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:26.692314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028094+0000","last_change":"2024-03-23T07:56:22.023659+0000","last_active":"2024-03-23T07:56:22.028094+0000","last_peered":"2024-03-23T07:56:22.028094+0000","last_clean":"2024-03-23T07:56:22.028094+0000","last_became_active":"2024-03-23T07:56:22.023230+0000","last_became_peered":"2024-03-23T07:56:22.023230+0000","last_unstale":"2024-03-23T07:56:22.028094+0000","last_undegraded":"2024-03-23T07:56:22.028094+0000","last_fullsized":"2024-03-23T07:56:22.028094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002303+0000","last_change":"2024-03-23T07:56:17.729587+0000","last_active":"2024-03-23T07:56:22.002303+0000","last_peered":"2024-03-23T07:56:22.002303+0000","last_clean":"2024-03-23T07:56:22.002303+0000","last_became_active":"2024-03-23T07:56:17.727873+0000","last_became_peered":"2024-03-23T07:56:17.727873+0000","last_unstale":"2024-03-23T07:56:22.002303+0000","last_undegraded":"2024-03-23T07:56:22.002303+0000","last_fullsized":"2024-03-23T07:56:22.002303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017227+0000","last_change":"2024-03-23T07:56:17.215408+0000","last_active":"2024-03-23T07:56:22.017227+0000","last_peered":"2024-03-23T07:56:22.017227+0000","last_clean":"2024-03-23T07:56:22.017227+0000","last_became_active":"2024-03-23T07:56:17.215155+0000","last_became_peered":"2024-03-23T07:56:17.215155+0000","last_unstale":"2024-03-23T07:56:22.017227+0000","last_undegraded":"2024-03-23T07:56:22.017227+0000","last_fullsized":"2024-03-23T07:56:22.017227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097238+0000","last_change":"2024-03-23T07:56:22.010493+0000","last_active":"2024-03-23T07:56:22.097238+0000","last_peered":"2024-03-23T07:56:22.097238+0000","last_clean":"2024-03-23T07:56:22.097238+0000","last_became_active":"2024-03-23T07:56:22.010048+0000","last_became_peered":"2024-03-23T07:56:22.010048+0000","last_unstale":"2024-03-23T07:56:22.097238+0000","last_undegraded":"2024-03-23T07:56:22.097238+0000","last_fullsized":"2024-03-23T07:56:22.097238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:17.490111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096554+0000","last_change":"2024-03-23T07:56:17.735587+0000","last_active":"2024-03-23T07:56:22.096554+0000","last_peered":"2024-03-23T07:56:22.096554+0000","last_clean":"2024-03-23T07:56:22.096554+0000","last_became_active":"2024-03-23T07:56:17.735319+0000","last_became_peered":"2024-03-23T07:56:17.735319+0000","last_unstale":"2024-03-23T07:56:22.096554+0000","last_undegraded":"2024-03-23T07:56:22.096554+0000","last_fullsized":"2024-03-23T07:56:22.096554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:36.310952+0000","last_change":"2024-03-23T07:56:17.217671+0000","last_active":"2024-03-23T07:56:36.310952+0000","last_peered":"2024-03-23T07:56:36.310952+0000","last_clean":"2024-03-23T07:56:36.310952+0000","last_became_active":"2024-03-23T07:56:17.217332+0000","last_became_peered":"2024-03-23T07:56:17.217332+0000","last_unstale":"2024-03-23T07:56:36.310952+0000","last_undegraded":"2024-03-23T07:56:36.310952+0000","last_fullsized":"2024-03-23T07:56:36.310952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:40:30.911275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018197+0000","last_change":"2024-03-23T07:56:22.013208+0000","last_active":"2024-03-23T07:56:22.018197+0000","last_peered":"2024-03-23T07:56:22.018197+0000","last_clean":"2024-03-23T07:56:22.018197+0000","last_became_active":"2024-03-23T07:56:22.012577+0000","last_became_peered":"2024-03-23T07:56:22.012577+0000","last_unstale":"2024-03-23T07:56:22.018197+0000","last_undegraded":"2024-03-23T07:56:22.018197+0000","last_fullsized":"2024-03-23T07:56:22.018197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097206+0000","last_change":"2024-03-23T07:56:17.735251+0000","last_active":"2024-03-23T07:56:22.097206+0000","last_peered":"2024-03-23T07:56:22.097206+0000","last_clean":"2024-03-23T07:56:22.097206+0000","last_became_active":"2024-03-23T07:56:17.731173+0000","last_became_peered":"2024-03-23T07:56:17.731173+0000","last_unstale":"2024-03-23T07:56:22.097206+0000","last_undegraded":"2024-03-23T07:56:22.097206+0000","last_fullsized":"2024-03-23T07:56:22.097206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:54.379935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018228+0000","last_change":"2024-03-23T07:56:17.216512+0000","last_active":"2024-03-23T07:56:22.018228+0000","last_peered":"2024-03-23T07:56:22.018228+0000","last_clean":"2024-03-23T07:56:22.018228+0000","last_became_active":"2024-03-23T07:56:17.215743+0000","last_became_peered":"2024-03-23T07:56:17.215743+0000","last_unstale":"2024-03-23T07:56:22.018228+0000","last_undegraded":"2024-03-23T07:56:22.018228+0000","last_fullsized":"2024-03-23T07:56:22.018228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093611+0000","last_change":"2024-03-23T07:56:22.014702+0000","last_active":"2024-03-23T07:56:22.093611+0000","last_peered":"2024-03-23T07:56:22.093611+0000","last_clean":"2024-03-23T07:56:22.093611+0000","last_became_active":"2024-03-23T07:56:22.014290+0000","last_became_peered":"2024-03-23T07:56:22.014290+0000","last_unstale":"2024-03-23T07:56:22.093611+0000","last_undegraded":"2024-03-23T07:56:22.093611+0000","last_fullsized":"2024-03-23T07:56:22.093611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005107+0000","last_change":"2024-03-23T07:56:17.725552+0000","last_active":"2024-03-23T07:56:22.005107+0000","last_peered":"2024-03-23T07:56:22.005107+0000","last_clean":"2024-03-23T07:56:22.005107+0000","last_became_active":"2024-03-23T07:56:17.725241+0000","last_became_peered":"2024-03-23T07:56:17.725241+0000","last_unstale":"2024-03-23T07:56:22.005107+0000","last_undegraded":"2024-03-23T07:56:22.005107+0000","last_fullsized":"2024-03-23T07:56:22.005107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:08:14.226813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.999373+0000","last_change":"2024-03-23T07:56:17.224220+0000","last_active":"2024-03-23T07:56:21.999373+0000","last_peered":"2024-03-23T07:56:21.999373+0000","last_clean":"2024-03-23T07:56:21.999373+0000","last_became_active":"2024-03-23T07:56:17.223476+0000","last_became_peered":"2024-03-23T07:56:17.223476+0000","last_unstale":"2024-03-23T07:56:21.999373+0000","last_undegraded":"2024-03-23T07:56:21.999373+0000","last_fullsized":"2024-03-23T07:56:21.999373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031049+0000","last_change":"2024-03-23T07:56:22.027325+0000","last_active":"2024-03-23T07:56:22.031049+0000","last_peered":"2024-03-23T07:56:22.031049+0000","last_clean":"2024-03-23T07:56:22.031049+0000","last_became_active":"2024-03-23T07:56:22.026858+0000","last_became_peered":"2024-03-23T07:56:22.026858+0000","last_unstale":"2024-03-23T07:56:22.031049+0000","last_undegraded":"2024-03-23T07:56:22.031049+0000","last_fullsized":"2024-03-23T07:56:22.031049+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:38.880449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093719+0000","last_change":"2024-03-23T07:56:17.736540+0000","last_active":"2024-03-23T07:56:22.093719+0000","last_peered":"2024-03-23T07:56:22.093719+0000","last_clean":"2024-03-23T07:56:22.093719+0000","last_became_active":"2024-03-23T07:56:17.727131+0000","last_became_peered":"2024-03-23T07:56:17.727131+0000","last_unstale":"2024-03-23T07:56:22.093719+0000","last_undegraded":"2024-03-23T07:56:22.093719+0000","last_fullsized":"2024-03-23T07:56:22.093719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.006955+0000","last_change":"2024-03-23T07:56:17.215558+0000","last_active":"2024-03-23T07:56:22.006955+0000","last_peered":"2024-03-23T07:56:22.006955+0000","last_clean":"2024-03-23T07:56:22.006955+0000","last_became_active":"2024-03-23T07:56:17.214342+0000","last_became_peered":"2024-03-23T07:56:17.214342+0000","last_unstale":"2024-03-23T07:56:22.006955+0000","last_undegraded":"2024-03-23T07:56:22.006955+0000","last_fullsized":"2024-03-23T07:56:22.006955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.010553+0000","last_change":"2024-03-23T07:56:22.008000+0000","last_active":"2024-03-23T07:56:22.010553+0000","last_peered":"2024-03-23T07:56:22.010553+0000","last_clean":"2024-03-23T07:56:22.010553+0000","last_became_active":"2024-03-23T07:56:22.006996+0000","last_became_peered":"2024-03-23T07:56:22.006996+0000","last_unstale":"2024-03-23T07:56:22.010553+0000","last_undegraded":"2024-03-23T07:56:22.010553+0000","last_fullsized":"2024-03-23T07:56:22.010553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:24:55.446700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015561+0000","last_change":"2024-03-23T07:56:22.009204+0000","last_active":"2024-03-23T07:56:22.015561+0000","last_peered":"2024-03-23T07:56:22.015561+0000","last_clean":"2024-03-23T07:56:22.015561+0000","last_became_active":"2024-03-23T07:56:22.008744+0000","last_became_peered":"2024-03-23T07:56:22.008744+0000","last_unstale":"2024-03-23T07:56:22.015561+0000","last_undegraded":"2024-03-23T07:56:22.015561+0000","last_fullsized":"2024-03-23T07:56:22.015561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:28.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984217+0000","last_change":"2024-03-23T07:56:17.733961+0000","last_active":"2024-03-23T07:56:21.984217+0000","last_peered":"2024-03-23T07:56:21.984217+0000","last_clean":"2024-03-23T07:56:21.984217+0000","last_became_active":"2024-03-23T07:56:17.733646+0000","last_became_peered":"2024-03-23T07:56:17.733646+0000","last_unstale":"2024-03-23T07:56:21.984217+0000","last_undegraded":"2024-03-23T07:56:21.984217+0000","last_fullsized":"2024-03-23T07:56:21.984217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002413+0000","last_change":"2024-03-23T07:56:17.223808+0000","last_active":"2024-03-23T07:56:22.002413+0000","last_peered":"2024-03-23T07:56:22.002413+0000","last_clean":"2024-03-23T07:56:22.002413+0000","last_became_active":"2024-03-23T07:56:17.223429+0000","last_became_peered":"2024-03-23T07:56:17.223429+0000","last_unstale":"2024-03-23T07:56:22.002413+0000","last_undegraded":"2024-03-23T07:56:22.002413+0000","last_fullsized":"2024-03-23T07:56:22.002413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:57.800903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097333+0000","last_change":"2024-03-23T07:56:22.015838+0000","last_active":"2024-03-23T07:56:22.097333+0000","last_peered":"2024-03-23T07:56:22.097333+0000","last_clean":"2024-03-23T07:56:22.097333+0000","last_became_active":"2024-03-23T07:56:22.015393+0000","last_became_peered":"2024-03-23T07:56:22.015393+0000","last_unstale":"2024-03-23T07:56:22.097333+0000","last_undegraded":"2024-03-23T07:56:22.097333+0000","last_fullsized":"2024-03-23T07:56:22.097333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:35:13.840057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985820+0000","last_change":"2024-03-23T07:56:17.729460+0000","last_active":"2024-03-23T07:56:21.985820+0000","last_peered":"2024-03-23T07:56:21.985820+0000","last_clean":"2024-03-23T07:56:21.985820+0000","last_became_active":"2024-03-23T07:56:17.728590+0000","last_became_peered":"2024-03-23T07:56:17.728590+0000","last_unstale":"2024-03-23T07:56:21.985820+0000","last_undegraded":"2024-03-23T07:56:21.985820+0000","last_fullsized":"2024-03-23T07:56:21.985820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:32:18.762479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001880+0000","last_change":"2024-03-23T07:56:17.216006+0000","last_active":"2024-03-23T07:56:22.001880+0000","last_peered":"2024-03-23T07:56:22.001880+0000","last_clean":"2024-03-23T07:56:22.001880+0000","last_became_active":"2024-03-23T07:56:17.215734+0000","last_became_peered":"2024-03-23T07:56:17.215734+0000","last_unstale":"2024-03-23T07:56:22.001880+0000","last_undegraded":"2024-03-23T07:56:22.001880+0000","last_fullsized":"2024-03-23T07:56:22.001880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003800+0000","last_change":"2024-03-23T07:56:17.225732+0000","last_active":"2024-03-23T07:56:22.003800+0000","last_peered":"2024-03-23T07:56:22.003800+0000","last_clean":"2024-03-23T07:56:22.003800+0000","last_became_active":"2024-03-23T07:56:17.225216+0000","last_became_peered":"2024-03-23T07:56:17.225216+0000","last_unstale":"2024-03-23T07:56:22.003800+0000","last_undegraded":"2024-03-23T07:56:22.003800+0000","last_fullsized":"2024-03-23T07:56:22.003800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:08.861374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982688+0000","last_change":"2024-03-23T07:56:17.750834+0000","last_active":"2024-03-23T07:56:21.982688+0000","last_peered":"2024-03-23T07:56:21.982688+0000","last_clean":"2024-03-23T07:56:21.982688+0000","last_became_active":"2024-03-23T07:56:17.750483+0000","last_became_peered":"2024-03-23T07:56:17.750483+0000","last_unstale":"2024-03-23T07:56:21.982688+0000","last_undegraded":"2024-03-23T07:56:21.982688+0000","last_fullsized":"2024-03-23T07:56:21.982688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018371+0000","last_change":"2024-03-23T07:56:22.013558+0000","last_active":"2024-03-23T07:56:22.018371+0000","last_peered":"2024-03-23T07:56:22.018371+0000","last_clean":"2024-03-23T07:56:22.018371+0000","last_became_active":"2024-03-23T07:56:22.011671+0000","last_became_peered":"2024-03-23T07:56:22.011671+0000","last_unstale":"2024-03-23T07:56:22.018371+0000","last_undegraded":"2024-03-23T07:56:22.018371+0000","last_fullsized":"2024-03-23T07:56:22.018371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:07.197610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991122+0000","last_change":"2024-03-23T07:56:17.217610+0000","last_active":"2024-03-23T07:56:21.991122+0000","last_peered":"2024-03-23T07:56:21.991122+0000","last_clean":"2024-03-23T07:56:21.991122+0000","last_became_active":"2024-03-23T07:56:17.217342+0000","last_became_peered":"2024-03-23T07:56:17.217342+0000","last_unstale":"2024-03-23T07:56:21.991122+0000","last_undegraded":"2024-03-23T07:56:21.991122+0000","last_fullsized":"2024-03-23T07:56:21.991122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:27.578107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093588+0000","last_change":"2024-03-23T07:56:17.719621+0000","last_active":"2024-03-23T07:56:22.093588+0000","last_peered":"2024-03-23T07:56:22.093588+0000","last_clean":"2024-03-23T07:56:22.093588+0000","last_became_active":"2024-03-23T07:56:17.719296+0000","last_became_peered":"2024-03-23T07:56:17.719296+0000","last_unstale":"2024-03-23T07:56:22.093588+0000","last_undegraded":"2024-03-23T07:56:22.093588+0000","last_fullsized":"2024-03-23T07:56:22.093588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031093+0000","last_change":"2024-03-23T07:56:22.027765+0000","last_active":"2024-03-23T07:56:22.031093+0000","last_peered":"2024-03-23T07:56:22.031093+0000","last_clean":"2024-03-23T07:56:22.031093+0000","last_became_active":"2024-03-23T07:56:22.027420+0000","last_became_peered":"2024-03-23T07:56:22.027420+0000","last_unstale":"2024-03-23T07:56:22.031093+0000","last_undegraded":"2024-03-23T07:56:22.031093+0000","last_fullsized":"2024-03-23T07:56:22.031093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:25.578708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093412+0000","last_change":"2024-03-23T07:56:17.721639+0000","last_active":"2024-03-23T07:56:22.093412+0000","last_peered":"2024-03-23T07:56:22.093412+0000","last_clean":"2024-03-23T07:56:22.093412+0000","last_became_active":"2024-03-23T07:56:17.721372+0000","last_became_peered":"2024-03-23T07:56:17.721372+0000","last_unstale":"2024-03-23T07:56:22.093412+0000","last_undegraded":"2024-03-23T07:56:22.093412+0000","last_fullsized":"2024-03-23T07:56:22.093412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014858+0000","last_change":"2024-03-23T07:56:17.741387+0000","last_active":"2024-03-23T07:56:22.014858+0000","last_peered":"2024-03-23T07:56:22.014858+0000","last_clean":"2024-03-23T07:56:22.014858+0000","last_became_active":"2024-03-23T07:56:17.741091+0000","last_became_peered":"2024-03-23T07:56:17.741091+0000","last_unstale":"2024-03-23T07:56:22.014858+0000","last_undegraded":"2024-03-23T07:56:22.014858+0000","last_fullsized":"2024-03-23T07:56:22.014858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097117+0000","last_change":"2024-03-23T07:56:17.729701+0000","last_active":"2024-03-23T07:56:22.097117+0000","last_peered":"2024-03-23T07:56:22.097117+0000","last_clean":"2024-03-23T07:56:22.097117+0000","last_became_active":"2024-03-23T07:56:17.729481+0000","last_became_peered":"2024-03-23T07:56:17.729481+0000","last_unstale":"2024-03-23T07:56:22.097117+0000","last_undegraded":"2024-03-23T07:56:22.097117+0000","last_fullsized":"2024-03-23T07:56:22.097117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.990887+0000","last_change":"2024-03-23T07:56:17.724914+0000","last_active":"2024-03-23T07:56:21.990887+0000","last_peered":"2024-03-23T07:56:21.990887+0000","last_clean":"2024-03-23T07:56:21.990887+0000","last_became_active":"2024-03-23T07:56:17.724627+0000","last_became_peered":"2024-03-23T07:56:17.724627+0000","last_unstale":"2024-03-23T07:56:21.990887+0000","last_undegraded":"2024-03-23T07:56:21.990887+0000","last_fullsized":"2024-03-23T07:56:21.990887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:36.852829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002999+0000","last_change":"2024-03-23T07:56:17.745053+0000","last_active":"2024-03-23T07:56:22.002999+0000","last_peered":"2024-03-23T07:56:22.002999+0000","last_clean":"2024-03-23T07:56:22.002999+0000","last_became_active":"2024-03-23T07:56:17.744775+0000","last_became_peered":"2024-03-23T07:56:17.744775+0000","last_unstale":"2024-03-23T07:56:22.002999+0000","last_undegraded":"2024-03-23T07:56:22.002999+0000","last_fullsized":"2024-03-23T07:56:22.002999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001976+0000","last_change":"2024-03-23T07:56:17.726849+0000","last_active":"2024-03-23T07:56:22.001976+0000","last_peered":"2024-03-23T07:56:22.001976+0000","last_clean":"2024-03-23T07:56:22.001976+0000","last_became_active":"2024-03-23T07:56:17.726339+0000","last_became_peered":"2024-03-23T07:56:17.726339+0000","last_unstale":"2024-03-23T07:56:22.001976+0000","last_undegraded":"2024-03-23T07:56:22.001976+0000","last_fullsized":"2024-03-23T07:56:22.001976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:19:42.241859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093701+0000","last_change":"2024-03-23T07:56:17.736674+0000","last_active":"2024-03-23T07:56:22.093701+0000","last_peered":"2024-03-23T07:56:22.093701+0000","last_clean":"2024-03-23T07:56:22.093701+0000","last_became_active":"2024-03-23T07:56:17.728841+0000","last_became_peered":"2024-03-23T07:56:17.728841+0000","last_unstale":"2024-03-23T07:56:22.093701+0000","last_undegraded":"2024-03-23T07:56:22.093701+0000","last_fullsized":"2024-03-23T07:56:22.093701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000680+0000","last_change":"2024-03-23T07:56:17.734249+0000","last_active":"2024-03-23T07:56:22.000680+0000","last_peered":"2024-03-23T07:56:22.000680+0000","last_clean":"2024-03-23T07:56:22.000680+0000","last_became_active":"2024-03-23T07:56:17.733805+0000","last_became_peered":"2024-03-23T07:56:17.733805+0000","last_unstale":"2024-03-23T07:56:22.000680+0000","last_undegraded":"2024-03-23T07:56:22.000680+0000","last_fullsized":"2024-03-23T07:56:22.000680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:47.287496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097031+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097031+0000","last_peered":"2024-03-23T07:56:22.097031+0000","last_clean":"2024-03-23T07:56:22.097031+0000","last_became_active":"2024-03-23T07:56:17.727006+0000","last_became_peered":"2024-03-23T07:56:17.727006+0000","last_unstale":"2024-03-23T07:56:22.097031+0000","last_undegraded":"2024-03-23T07:56:22.097031+0000","last_fullsized":"2024-03-23T07:56:22.097031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:54.056625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.989132+0000","last_change":"2024-03-23T07:56:17.730066+0000","last_active":"2024-03-23T07:56:21.989132+0000","last_peered":"2024-03-23T07:56:21.989132+0000","last_clean":"2024-03-23T07:56:21.989132+0000","last_became_active":"2024-03-23T07:56:17.729250+0000","last_became_peered":"2024-03-23T07:56:17.729250+0000","last_unstale":"2024-03-23T07:56:21.989132+0000","last_undegraded":"2024-03-23T07:56:21.989132+0000","last_fullsized":"2024-03-23T07:56:21.989132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:28.507099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093550+0000","last_change":"2024-03-23T07:56:17.732206+0000","last_active":"2024-03-23T07:56:22.093550+0000","last_peered":"2024-03-23T07:56:22.093550+0000","last_clean":"2024-03-23T07:56:22.093550+0000","last_became_active":"2024-03-23T07:56:17.731391+0000","last_became_peered":"2024-03-23T07:56:17.731391+0000","last_unstale":"2024-03-23T07:56:22.093550+0000","last_undegraded":"2024-03-23T07:56:22.093550+0000","last_fullsized":"2024-03-23T07:56:22.093550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982447+0000","last_change":"2024-03-23T07:56:17.727582+0000","last_active":"2024-03-23T07:56:21.982447+0000","last_peered":"2024-03-23T07:56:21.982447+0000","last_clean":"2024-03-23T07:56:21.982447+0000","last_became_active":"2024-03-23T07:56:17.727312+0000","last_became_peered":"2024-03-23T07:56:17.727312+0000","last_unstale":"2024-03-23T07:56:21.982447+0000","last_undegraded":"2024-03-23T07:56:21.982447+0000","last_fullsized":"2024-03-23T07:56:21.982447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096494+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096494+0000","last_peered":"2024-03-23T07:56:22.096494+0000","last_clean":"2024-03-23T07:56:22.096494+0000","last_became_active":"2024-03-23T07:56:17.743717+0000","last_became_peered":"2024-03-23T07:56:17.743717+0000","last_unstale":"2024-03-23T07:56:22.096494+0000","last_undegraded":"2024-03-23T07:56:22.096494+0000","last_fullsized":"2024-03-23T07:56:22.096494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:35.095859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097297+0000","last_change":"2024-03-23T07:56:17.729192+0000","last_active":"2024-03-23T07:56:22.097297+0000","last_peered":"2024-03-23T07:56:22.097297+0000","last_clean":"2024-03-23T07:56:22.097297+0000","last_became_active":"2024-03-23T07:56:17.728922+0000","last_became_peered":"2024-03-23T07:56:17.728922+0000","last_unstale":"2024-03-23T07:56:22.097297+0000","last_undegraded":"2024-03-23T07:56:22.097297+0000","last_fullsized":"2024-03-23T07:56:22.097297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017348+0000","last_change":"2024-03-23T07:56:17.731099+0000","last_active":"2024-03-23T07:56:22.017348+0000","last_peered":"2024-03-23T07:56:22.017348+0000","last_clean":"2024-03-23T07:56:22.017348+0000","last_became_active":"2024-03-23T07:56:17.730597+0000","last_became_peered":"2024-03-23T07:56:17.730597+0000","last_unstale":"2024-03-23T07:56:22.017348+0000","last_undegraded":"2024-03-23T07:56:22.017348+0000","last_fullsized":"2024-03-23T07:56:22.017348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097349+0000","last_change":"2024-03-23T07:56:17.748094+0000","last_active":"2024-03-23T07:56:22.097349+0000","last_peered":"2024-03-23T07:56:22.097349+0000","last_clean":"2024-03-23T07:56:22.097349+0000","last_became_active":"2024-03-23T07:56:17.747867+0000","last_became_peered":"2024-03-23T07:56:17.747867+0000","last_unstale":"2024-03-23T07:56:22.097349+0000","last_undegraded":"2024-03-23T07:56:22.097349+0000","last_fullsized":"2024-03-23T07:56:22.097349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:53.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097004+0000","last_change":"2024-03-23T07:56:17.731875+0000","last_active":"2024-03-23T07:56:22.097004+0000","last_peered":"2024-03-23T07:56:22.097004+0000","last_clean":"2024-03-23T07:56:22.097004+0000","last_became_active":"2024-03-23T07:56:17.731600+0000","last_became_peered":"2024-03-23T07:56:17.731600+0000","last_unstale":"2024-03-23T07:56:22.097004+0000","last_undegraded":"2024-03-23T07:56:22.097004+0000","last_fullsized":"2024-03-23T07:56:22.097004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:55.441592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988753+0000","last_change":"2024-03-23T07:56:17.724024+0000","last_active":"2024-03-23T07:56:21.988753+0000","last_peered":"2024-03-23T07:56:21.988753+0000","last_clean":"2024-03-23T07:56:21.988753+0000","last_became_active":"2024-03-23T07:56:17.723743+0000","last_became_peered":"2024-03-23T07:56:17.723743+0000","last_unstale":"2024-03-23T07:56:21.988753+0000","last_undegraded":"2024-03-23T07:56:21.988753+0000","last_fullsized":"2024-03-23T07:56:21.988753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:32:51.077761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002205+0000","last_change":"2024-03-23T07:56:17.723640+0000","last_active":"2024-03-23T07:56:22.002205+0000","last_peered":"2024-03-23T07:56:22.002205+0000","last_clean":"2024-03-23T07:56:22.002205+0000","last_became_active":"2024-03-23T07:56:17.723352+0000","last_became_peered":"2024-03-23T07:56:17.723352+0000","last_unstale":"2024-03-23T07:56:22.002205+0000","last_undegraded":"2024-03-23T07:56:22.002205+0000","last_fullsized":"2024-03-23T07:56:22.002205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:48.788785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002849+0000","last_change":"2024-03-23T07:56:17.745108+0000","last_active":"2024-03-23T07:56:22.002849+0000","last_peered":"2024-03-23T07:56:22.002849+0000","last_clean":"2024-03-23T07:56:22.002849+0000","last_became_active":"2024-03-23T07:56:17.744769+0000","last_became_peered":"2024-03-23T07:56:17.744769+0000","last_unstale":"2024-03-23T07:56:22.002849+0000","last_undegraded":"2024-03-23T07:56:22.002849+0000","last_fullsized":"2024-03-23T07:56:22.002849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:45.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096478+0000","last_change":"2024-03-23T07:56:17.743989+0000","last_active":"2024-03-23T07:56:22.096478+0000","last_peered":"2024-03-23T07:56:22.096478+0000","last_clean":"2024-03-23T07:56:22.096478+0000","last_became_active":"2024-03-23T07:56:17.743697+0000","last_became_peered":"2024-03-23T07:56:17.743697+0000","last_unstale":"2024-03-23T07:56:22.096478+0000","last_undegraded":"2024-03-23T07:56:22.096478+0000","last_fullsized":"2024-03-23T07:56:22.096478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:33:48.226604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096538+0000","last_change":"2024-03-23T07:56:17.737898+0000","last_active":"2024-03-23T07:56:22.096538+0000","last_peered":"2024-03-23T07:56:22.096538+0000","last_clean":"2024-03-23T07:56:22.096538+0000","last_became_active":"2024-03-23T07:56:17.737650+0000","last_became_peered":"2024-03-23T07:56:17.737650+0000","last_unstale":"2024-03-23T07:56:22.096538+0000","last_undegraded":"2024-03-23T07:56:22.096538+0000","last_fullsized":"2024-03-23T07:56:22.096538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:21.794118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096613+0000","last_change":"2024-03-23T07:56:17.741344+0000","last_active":"2024-03-23T07:56:22.096613+0000","last_peered":"2024-03-23T07:56:22.096613+0000","last_clean":"2024-03-23T07:56:22.096613+0000","last_became_active":"2024-03-23T07:56:17.740336+0000","last_became_peered":"2024-03-23T07:56:17.740336+0000","last_unstale":"2024-03-23T07:56:22.096613+0000","last_undegraded":"2024-03-23T07:56:22.096613+0000","last_fullsized":"2024-03-23T07:56:22.096613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093627+0000","last_change":"2024-03-23T07:56:17.725091+0000","last_active":"2024-03-23T07:56:22.093627+0000","last_peered":"2024-03-23T07:56:22.093627+0000","last_clean":"2024-03-23T07:56:22.093627+0000","last_became_active":"2024-03-23T07:56:17.724794+0000","last_became_peered":"2024-03-23T07:56:17.724794+0000","last_unstale":"2024-03-23T07:56:22.093627+0000","last_undegraded":"2024-03-23T07:56:22.093627+0000","last_fullsized":"2024-03-23T07:56:22.093627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983064+0000","last_change":"2024-03-23T07:56:17.749204+0000","last_active":"2024-03-23T07:56:21.983064+0000","last_peered":"2024-03-23T07:56:21.983064+0000","last_clean":"2024-03-23T07:56:21.983064+0000","last_became_active":"2024-03-23T07:56:17.748888+0000","last_became_peered":"2024-03-23T07:56:17.748888+0000","last_unstale":"2024-03-23T07:56:21.983064+0000","last_undegraded":"2024-03-23T07:56:21.983064+0000","last_fullsized":"2024-03-23T07:56:21.983064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:14.142988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096378+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096378+0000","last_peered":"2024-03-23T07:56:22.096378+0000","last_clean":"2024-03-23T07:56:22.096378+0000","last_became_active":"2024-03-23T07:56:17.743764+0000","last_became_peered":"2024-03-23T07:56:17.743764+0000","last_unstale":"2024-03-23T07:56:22.096378+0000","last_undegraded":"2024-03-23T07:56:22.096378+0000","last_fullsized":"2024-03-23T07:56:22.096378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093513+0000","last_change":"2024-03-23T07:56:17.724709+0000","last_active":"2024-03-23T07:56:22.093513+0000","last_peered":"2024-03-23T07:56:22.093513+0000","last_clean":"2024-03-23T07:56:22.093513+0000","last_became_active":"2024-03-23T07:56:17.724393+0000","last_became_peered":"2024-03-23T07:56:17.724393+0000","last_unstale":"2024-03-23T07:56:22.093513+0000","last_undegraded":"2024-03-23T07:56:22.093513+0000","last_fullsized":"2024-03-23T07:56:22.093513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096409+0000","last_change":"2024-03-23T07:56:17.742970+0000","last_active":"2024-03-23T07:56:22.096409+0000","last_peered":"2024-03-23T07:56:22.096409+0000","last_clean":"2024-03-23T07:56:22.096409+0000","last_became_active":"2024-03-23T07:56:17.742629+0000","last_became_peered":"2024-03-23T07:56:17.742629+0000","last_unstale":"2024-03-23T07:56:22.096409+0000","last_undegraded":"2024-03-23T07:56:22.096409+0000","last_fullsized":"2024-03-23T07:56:22.096409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:21:07.705635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093633+0000","last_change":"2024-03-23T07:56:17.736446+0000","last_active":"2024-03-23T07:56:22.093633+0000","last_peered":"2024-03-23T07:56:22.093633+0000","last_clean":"2024-03-23T07:56:22.093633+0000","last_became_active":"2024-03-23T07:56:17.727700+0000","last_became_peered":"2024-03-23T07:56:17.727700+0000","last_unstale":"2024-03-23T07:56:22.093633+0000","last_undegraded":"2024-03-23T07:56:22.093633+0000","last_fullsized":"2024-03-23T07:56:22.093633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991994+0000","last_change":"2024-03-23T07:56:17.732549+0000","last_active":"2024-03-23T07:56:21.991994+0000","last_peered":"2024-03-23T07:56:21.991994+0000","last_clean":"2024-03-23T07:56:21.991994+0000","last_became_active":"2024-03-23T07:56:17.732276+0000","last_became_peered":"2024-03-23T07:56:17.732276+0000","last_unstale":"2024-03-23T07:56:21.991994+0000","last_undegraded":"2024-03-23T07:56:21.991994+0000","last_fullsized":"2024-03-23T07:56:21.991994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:48.012231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000782+0000","last_change":"2024-03-23T07:56:17.732034+0000","last_active":"2024-03-23T07:56:22.000782+0000","last_peered":"2024-03-23T07:56:22.000782+0000","last_clean":"2024-03-23T07:56:22.000782+0000","last_became_active":"2024-03-23T07:56:17.731730+0000","last_became_peered":"2024-03-23T07:56:17.731730+0000","last_unstale":"2024-03-23T07:56:22.000782+0000","last_undegraded":"2024-03-23T07:56:22.000782+0000","last_fullsized":"2024-03-23T07:56:22.000782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:59:20.863874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096639+0000","last_change":"2024-03-23T07:56:17.746986+0000","last_active":"2024-03-23T07:56:22.096639+0000","last_peered":"2024-03-23T07:56:22.096639+0000","last_clean":"2024-03-23T07:56:22.096639+0000","last_became_active":"2024-03-23T07:56:17.746621+0000","last_became_peered":"2024-03-23T07:56:17.746621+0000","last_unstale":"2024-03-23T07:56:22.096639+0000","last_undegraded":"2024-03-23T07:56:22.096639+0000","last_fullsized":"2024-03-23T07:56:22.096639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:22.719041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476747,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]}]},{"osd":2,"up_from":16,"seq":68719476747,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":16,"seq":68719476747,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":694448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476749,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.482}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":108477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6379999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:56:52.063 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:56:52.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.297+0000 7f9fb9a00640 1 Processor -- start 2024-03-23T07:56:52.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.297+0000 7f9fb9a00640 1 -- start start 2024-03-23T07:56:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb9a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fb4156e30 0x7f9fb406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb40cb540 0x7f9fb40b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb9a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40cc8c0 con 0x7f9fb40b3310 2024-03-23T07:56:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb9a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb406d630 con 0x7f9fb4156e30 2024-03-23T07:56:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb415ce60 con 0x7f9fb40cb540 2024-03-23T07:56:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fb4156e30 0x7f9fb406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fb4156e30 0x7f9fb406d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33740/0 (socket says 172.21.15.105:33740) 2024-03-23T07:56:52.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb3e00640 1 -- 172.21.15.105:0/2506403549 learned_addr learned my addr 172.21.15.105:0/2506403549 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:52.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb9000640 1 --2- 172.21.15.105:0/2506403549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb40cb540 0x7f9fb40b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb3e00640 1 -- 172.21.15.105:0/2506403549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb40cb540 msgr2=0x7f9fb40b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:52.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb3e00640 1 --2- 172.21.15.105:0/2506403549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb40cb540 0x7f9fb40b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb3e00640 1 -- 172.21.15.105:0/2506403549 >> v1:172.21.15.132:6789/0 conn(0x7f9fb40b3310 legacy=0x7f9fb40cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:52.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.299+0000 7f9fb3e00640 1 -- 172.21.15.105:0/2506403549 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb415d120 con 0x7f9fb4156e30 2024-03-23T07:56:52.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.300+0000 7f9fb3e00640 1 --2- 172.21.15.105:0/2506403549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4156e30 0x7f9fb406d0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009e50 tx=0x7f9fa402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bde11d1115a6e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:52.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb2a00640 1 -- 172.21.15.105:0/2506403549 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa402ca70 con 0x7f9fb4156e30 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb2a00640 1 -- 172.21.15.105:0/2506403549 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa402cbd0 con 0x7f9fb4156e30 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb2a00640 1 -- 172.21.15.105:0/2506403549 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40343f0 con 0x7f9fb4156e30 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb9a00640 1 -- 172.21.15.105:0/2506403549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4156e30 msgr2=0x7f9fb406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/2506403549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4156e30 0x7f9fb406d0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009e50 tx=0x7f9fa402bed0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb9a00640 1 -- 172.21.15.105:0/2506403549 shutdown_connections 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/2506403549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb40cb540 0x7f9fb40b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/2506403549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4156e30 0x7f9fb406d0f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.301+0000 7f9fb9a00640 1 -- 172.21.15.105:0/2506403549 >> 172.21.15.105:0/2506403549 conn(0x7f9fb4065df0 msgr2=0x7f9fb4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.302+0000 7f9fb9a00640 1 -- 172.21.15.105:0/2506403549 shutdown_connections 2024-03-23T07:56:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.302+0000 7f9fb9a00640 1 -- 172.21.15.105:0/2506403549 wait complete. 2024-03-23T07:56:52.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.303+0000 7f9fb9a00640 1 Processor -- start 2024-03-23T07:56:52.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.304+0000 7f9fb9a00640 1 -- start start 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb9a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fb40b3310 0x7f9fb413d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb9a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb40cb540 0x7f9fb413e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb4156e30 0x7f9fb40aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb9a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb414e270 con 0x7f9fb40cb540 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb9a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb413dd90 con 0x7f9fb40b3310 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40afa40 con 0x7f9fb4156e30 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb40cb540 0x7f9fb413e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb40cb540 0x7f9fb413e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33756/0 (socket says 172.21.15.105:33756) 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3400640 1 -- 172.21.15.105:0/953875923 learned_addr learned my addr 172.21.15.105:0/953875923 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3e00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fb40b3310 0x7f9fb413d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb9000640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb4156e30 0x7f9fb40aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3400640 1 -- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb4156e30 msgr2=0x7f9fb40aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3400640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb4156e30 0x7f9fb40aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3400640 1 -- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fb40b3310 msgr2=0x7f9fb413d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.305+0000 7f9fb3400640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fb40b3310 0x7f9fb413d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.306+0000 7f9fb3400640 1 -- 172.21.15.105:0/953875923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4009b30 con 0x7f9fb40cb540 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.306+0000 7f9fb3400640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb40cb540 0x7f9fb413e000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c400 tx=0x7f9fa000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.306+0000 7f9fabe00640 1 -- 172.21.15.105:0/953875923 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0011790 con 0x7f9fb40cb540 2024-03-23T07:56:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.306+0000 7f9fabe00640 1 -- 172.21.15.105:0/953875923 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa0011930 con 0x7f9fb40cb540 2024-03-23T07:56:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.306+0000 7f9fabe00640 1 -- 172.21.15.105:0/953875923 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0011ce0 con 0x7f9fb40cb540 2024-03-23T07:56:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.306+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb413e540 con 0x7f9fb40cb540 2024-03-23T07:56:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.306+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb4150980 con 0x7f9fb40cb540 2024-03-23T07:56:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.307+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb406d800 con 0x7f9fb40cb540 2024-03-23T07:56:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.308+0000 7f9fabe00640 1 -- 172.21.15.105:0/953875923 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9fa001c460 con 0x7f9fb40cb540 2024-03-23T07:56:52.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.310+0000 7f9fabe00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f9406ac00 0x7f9f9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.310+0000 7f9fabe00640 1 -- 172.21.15.105:0/953875923 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9fa0089560 con 0x7f9fb40cb540 2024-03-23T07:56:52.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.310+0000 7f9fb3e00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f9406ac00 0x7f9f9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.311+0000 7f9fb3e00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f9406ac00 0x7f9f9406d0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4035000 tx=0x7f9fa4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:52.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.314+0000 7f9fabe00640 1 -- 172.21.15.105:0/953875923 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9fa00544e0 con 0x7f9fb40cb540 2024-03-23T07:56:52.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.490+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9fb415d6e0 con 0x7f9f9406ac00 2024-03-23T07:56:52.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.516+0000 7f9fabe00640 1 -- 172.21.15.105:0/953875923 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345042 (secure 0 0 0) 0x7f9fa4044090 con 0x7f9f9406ac00 2024-03-23T07:56:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:56:52.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.522+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f9406ac00 msgr2=0x7f9f9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.522+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f9406ac00 0x7f9f9406d0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4035000 tx=0x7f9fa4030000 comp rx=0 tx=0).stop 2024-03-23T07:56:52.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.522+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb40cb540 msgr2=0x7f9fb413e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.522+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb40cb540 0x7f9fb413e000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c400 tx=0x7f9fa000c8c0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 shutdown_connections 2024-03-23T07:56:52.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f9406ac00 0x7f9f9406d0c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9fb4156e30 0x7f9fb40aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb40cb540 0x7f9fb413e000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 --2- 172.21.15.105:0/953875923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fb40b3310 0x7f9fb413d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 >> 172.21.15.105:0/953875923 conn(0x7f9fb4065df0 msgr2=0x7f9fb4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:52.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 shutdown_connections 2024-03-23T07:56:52.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.523+0000 7f9fb9a00640 1 -- 172.21.15.105:0/953875923 wait complete. 2024-03-23T07:56:52.541 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-03-23T07:56:52.052893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2022002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004846"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995336+0000","last_change":"2024-03-23T07:56:17.227981+0000","last_active":"2024-03-23T07:56:21.995336+0000","last_peered":"2024-03-23T07:56:21.995336+0000","last_clean":"2024-03-23T07:56:21.995336+0000","last_became_active":"2024-03-23T07:56:17.227576+0000","last_became_peered":"2024-03-23T07:56:17.227576+0000","last_unstale":"2024-03-23T07:56:21.995336+0000","last_undegraded":"2024-03-23T07:56:21.995336+0000","last_fullsized":"2024-03-23T07:56:21.995336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001380+0000","last_change":"2024-03-23T07:56:17.730054+0000","last_active":"2024-03-23T07:56:22.001380+0000","last_peered":"2024-03-23T07:56:22.001380+0000","last_clean":"2024-03-23T07:56:22.001380+0000","last_became_active":"2024-03-23T07:56:17.729766+0000","last_became_peered":"2024-03-23T07:56:17.729766+0000","last_unstale":"2024-03-23T07:56:22.001380+0000","last_undegraded":"2024-03-23T07:56:22.001380+0000","last_fullsized":"2024-03-23T07:56:22.001380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:14:39.987411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.032954+0000","last_change":"2024-03-23T07:56:22.029375+0000","last_active":"2024-03-23T07:56:22.032954+0000","last_peered":"2024-03-23T07:56:22.032954+0000","last_clean":"2024-03-23T07:56:22.032954+0000","last_became_active":"2024-03-23T07:56:22.028383+0000","last_became_peered":"2024-03-23T07:56:22.028383+0000","last_unstale":"2024-03-23T07:56:22.032954+0000","last_undegraded":"2024-03-23T07:56:22.032954+0000","last_fullsized":"2024-03-23T07:56:22.032954+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:06.760193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001542+0000","last_change":"2024-03-23T07:56:17.217630+0000","last_active":"2024-03-23T07:56:22.001542+0000","last_peered":"2024-03-23T07:56:22.001542+0000","last_clean":"2024-03-23T07:56:22.001542+0000","last_became_active":"2024-03-23T07:56:17.217179+0000","last_became_peered":"2024-03-23T07:56:17.217179+0000","last_unstale":"2024-03-23T07:56:22.001542+0000","last_undegraded":"2024-03-23T07:56:22.001542+0000","last_fullsized":"2024-03-23T07:56:22.001542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:27:59.587867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004329+0000","last_change":"2024-03-23T07:56:17.734047+0000","last_active":"2024-03-23T07:56:22.004329+0000","last_peered":"2024-03-23T07:56:22.004329+0000","last_clean":"2024-03-23T07:56:22.004329+0000","last_became_active":"2024-03-23T07:56:17.733482+0000","last_became_peered":"2024-03-23T07:56:17.733482+0000","last_unstale":"2024-03-23T07:56:22.004329+0000","last_undegraded":"2024-03-23T07:56:22.004329+0000","last_fullsized":"2024-03-23T07:56:22.004329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:47:35.800634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093379+0000","last_change":"2024-03-23T07:56:22.012905+0000","last_active":"2024-03-23T07:56:22.093379+0000","last_peered":"2024-03-23T07:56:22.093379+0000","last_clean":"2024-03-23T07:56:22.093379+0000","last_became_active":"2024-03-23T07:56:22.012340+0000","last_became_peered":"2024-03-23T07:56:22.012340+0000","last_unstale":"2024-03-23T07:56:22.093379+0000","last_undegraded":"2024-03-23T07:56:22.093379+0000","last_fullsized":"2024-03-23T07:56:22.093379+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:43.862034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000002+0000","last_change":"2024-03-23T07:56:17.224614+0000","last_active":"2024-03-23T07:56:22.000002+0000","last_peered":"2024-03-23T07:56:22.000002+0000","last_clean":"2024-03-23T07:56:22.000002+0000","last_became_active":"2024-03-23T07:56:17.223463+0000","last_became_peered":"2024-03-23T07:56:17.223463+0000","last_unstale":"2024-03-23T07:56:22.000002+0000","last_undegraded":"2024-03-23T07:56:22.000002+0000","last_fullsized":"2024-03-23T07:56:22.000002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093607+0000","last_change":"2024-03-23T07:56:17.729551+0000","last_active":"2024-03-23T07:56:22.093607+0000","last_peered":"2024-03-23T07:56:22.093607+0000","last_clean":"2024-03-23T07:56:22.093607+0000","last_became_active":"2024-03-23T07:56:17.729295+0000","last_became_peered":"2024-03-23T07:56:17.729295+0000","last_unstale":"2024-03-23T07:56:22.093607+0000","last_undegraded":"2024-03-23T07:56:22.093607+0000","last_fullsized":"2024-03-23T07:56:22.093607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093533+0000","last_change":"2024-03-23T07:56:22.005574+0000","last_active":"2024-03-23T07:56:22.093533+0000","last_peered":"2024-03-23T07:56:22.093533+0000","last_clean":"2024-03-23T07:56:22.093533+0000","last_became_active":"2024-03-23T07:56:22.005169+0000","last_became_peered":"2024-03-23T07:56:22.005169+0000","last_unstale":"2024-03-23T07:56:22.093533+0000","last_undegraded":"2024-03-23T07:56:22.093533+0000","last_fullsized":"2024-03-23T07:56:22.093533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005504+0000","last_change":"2024-03-23T07:56:17.223235+0000","last_active":"2024-03-23T07:56:22.005504+0000","last_peered":"2024-03-23T07:56:22.005504+0000","last_clean":"2024-03-23T07:56:22.005504+0000","last_became_active":"2024-03-23T07:56:17.222925+0000","last_became_peered":"2024-03-23T07:56:17.222925+0000","last_unstale":"2024-03-23T07:56:22.005504+0000","last_undegraded":"2024-03-23T07:56:22.005504+0000","last_fullsized":"2024-03-23T07:56:22.005504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097254+0000","last_change":"2024-03-23T07:56:17.745979+0000","last_active":"2024-03-23T07:56:22.097254+0000","last_peered":"2024-03-23T07:56:22.097254+0000","last_clean":"2024-03-23T07:56:22.097254+0000","last_became_active":"2024-03-23T07:56:17.745702+0000","last_became_peered":"2024-03-23T07:56:17.745702+0000","last_unstale":"2024-03-23T07:56:22.097254+0000","last_undegraded":"2024-03-23T07:56:22.097254+0000","last_fullsized":"2024-03-23T07:56:22.097254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:32:07.760652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093472+0000","last_change":"2024-03-23T07:56:22.021092+0000","last_active":"2024-03-23T07:56:22.093472+0000","last_peered":"2024-03-23T07:56:22.093472+0000","last_clean":"2024-03-23T07:56:22.093472+0000","last_became_active":"2024-03-23T07:56:22.017901+0000","last_became_peered":"2024-03-23T07:56:22.017901+0000","last_unstale":"2024-03-23T07:56:22.093472+0000","last_undegraded":"2024-03-23T07:56:22.093472+0000","last_fullsized":"2024-03-23T07:56:22.093472+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097129+0000","last_change":"2024-03-23T07:56:17.733964+0000","last_active":"2024-03-23T07:56:22.097129+0000","last_peered":"2024-03-23T07:56:22.097129+0000","last_clean":"2024-03-23T07:56:22.097129+0000","last_became_active":"2024-03-23T07:56:17.733721+0000","last_became_peered":"2024-03-23T07:56:17.733721+0000","last_unstale":"2024-03-23T07:56:22.097129+0000","last_undegraded":"2024-03-23T07:56:22.097129+0000","last_fullsized":"2024-03-23T07:56:22.097129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004014+0000","last_change":"2024-03-23T07:56:17.227903+0000","last_active":"2024-03-23T07:56:22.004014+0000","last_peered":"2024-03-23T07:56:22.004014+0000","last_clean":"2024-03-23T07:56:22.004014+0000","last_became_active":"2024-03-23T07:56:17.227224+0000","last_became_peered":"2024-03-23T07:56:17.227224+0000","last_unstale":"2024-03-23T07:56:22.004014+0000","last_undegraded":"2024-03-23T07:56:22.004014+0000","last_fullsized":"2024-03-23T07:56:22.004014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:46.265440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097221+0000","last_change":"2024-03-23T07:56:22.023790+0000","last_active":"2024-03-23T07:56:22.097221+0000","last_peered":"2024-03-23T07:56:22.097221+0000","last_clean":"2024-03-23T07:56:22.097221+0000","last_became_active":"2024-03-23T07:56:22.022870+0000","last_became_peered":"2024-03-23T07:56:22.022870+0000","last_unstale":"2024-03-23T07:56:22.097221+0000","last_undegraded":"2024-03-23T07:56:22.097221+0000","last_fullsized":"2024-03-23T07:56:22.097221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:40.283526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014342+0000","last_change":"2024-03-23T07:56:17.734095+0000","last_active":"2024-03-23T07:56:22.014342+0000","last_peered":"2024-03-23T07:56:22.014342+0000","last_clean":"2024-03-23T07:56:22.014342+0000","last_became_active":"2024-03-23T07:56:17.733771+0000","last_became_peered":"2024-03-23T07:56:17.733771+0000","last_unstale":"2024-03-23T07:56:22.014342+0000","last_undegraded":"2024-03-23T07:56:22.014342+0000","last_fullsized":"2024-03-23T07:56:22.014342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002387+0000","last_change":"2024-03-23T07:56:17.221331+0000","last_active":"2024-03-23T07:56:22.002387+0000","last_peered":"2024-03-23T07:56:22.002387+0000","last_clean":"2024-03-23T07:56:22.002387+0000","last_became_active":"2024-03-23T07:56:17.221062+0000","last_became_peered":"2024-03-23T07:56:17.221062+0000","last_unstale":"2024-03-23T07:56:22.002387+0000","last_undegraded":"2024-03-23T07:56:22.002387+0000","last_fullsized":"2024-03-23T07:56:22.002387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096460+0000","last_change":"2024-03-23T07:56:22.028190+0000","last_active":"2024-03-23T07:56:22.096460+0000","last_peered":"2024-03-23T07:56:22.096460+0000","last_clean":"2024-03-23T07:56:22.096460+0000","last_became_active":"2024-03-23T07:56:22.027188+0000","last_became_peered":"2024-03-23T07:56:22.027188+0000","last_unstale":"2024-03-23T07:56:22.096460+0000","last_undegraded":"2024-03-23T07:56:22.096460+0000","last_fullsized":"2024-03-23T07:56:22.096460+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985338+0000","last_change":"2024-03-23T07:56:17.728823+0000","last_active":"2024-03-23T07:56:21.985338+0000","last_peered":"2024-03-23T07:56:21.985338+0000","last_clean":"2024-03-23T07:56:21.985338+0000","last_became_active":"2024-03-23T07:56:17.728504+0000","last_became_peered":"2024-03-23T07:56:17.728504+0000","last_unstale":"2024-03-23T07:56:21.985338+0000","last_undegraded":"2024-03-23T07:56:21.985338+0000","last_fullsized":"2024-03-23T07:56:21.985338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:14:28.882493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984446+0000","last_change":"2024-03-23T07:56:17.209430+0000","last_active":"2024-03-23T07:56:21.984446+0000","last_peered":"2024-03-23T07:56:21.984446+0000","last_clean":"2024-03-23T07:56:21.984446+0000","last_became_active":"2024-03-23T07:56:17.207873+0000","last_became_peered":"2024-03-23T07:56:17.207873+0000","last_unstale":"2024-03-23T07:56:21.984446+0000","last_undegraded":"2024-03-23T07:56:21.984446+0000","last_fullsized":"2024-03-23T07:56:21.984446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.033429+0000","last_change":"2024-03-23T07:56:22.031440+0000","last_active":"2024-03-23T07:56:22.033429+0000","last_peered":"2024-03-23T07:56:22.033429+0000","last_clean":"2024-03-23T07:56:22.033429+0000","last_became_active":"2024-03-23T07:56:22.030403+0000","last_became_peered":"2024-03-23T07:56:22.030403+0000","last_unstale":"2024-03-23T07:56:22.033429+0000","last_undegraded":"2024-03-23T07:56:22.033429+0000","last_fullsized":"2024-03-23T07:56:22.033429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:40.289743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097084+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097084+0000","last_peered":"2024-03-23T07:56:22.097084+0000","last_clean":"2024-03-23T07:56:22.097084+0000","last_became_active":"2024-03-23T07:56:17.728164+0000","last_became_peered":"2024-03-23T07:56:17.728164+0000","last_unstale":"2024-03-23T07:56:22.097084+0000","last_undegraded":"2024-03-23T07:56:22.097084+0000","last_fullsized":"2024-03-23T07:56:22.097084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:11:45.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998174+0000","last_change":"2024-03-23T07:56:17.211209+0000","last_active":"2024-03-23T07:56:21.998174+0000","last_peered":"2024-03-23T07:56:21.998174+0000","last_clean":"2024-03-23T07:56:21.998174+0000","last_became_active":"2024-03-23T07:56:17.210792+0000","last_became_peered":"2024-03-23T07:56:17.210792+0000","last_unstale":"2024-03-23T07:56:21.998174+0000","last_undegraded":"2024-03-23T07:56:21.998174+0000","last_fullsized":"2024-03-23T07:56:21.998174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093682+0000","last_change":"2024-03-23T07:56:22.016254+0000","last_active":"2024-03-23T07:56:22.093682+0000","last_peered":"2024-03-23T07:56:22.093682+0000","last_clean":"2024-03-23T07:56:22.093682+0000","last_became_active":"2024-03-23T07:56:22.015890+0000","last_became_peered":"2024-03-23T07:56:22.015890+0000","last_unstale":"2024-03-23T07:56:22.093682+0000","last_undegraded":"2024-03-23T07:56:22.093682+0000","last_fullsized":"2024-03-23T07:56:22.093682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:15.477490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992721+0000","last_change":"2024-03-23T07:56:17.726007+0000","last_active":"2024-03-23T07:56:21.992721+0000","last_peered":"2024-03-23T07:56:21.992721+0000","last_clean":"2024-03-23T07:56:21.992721+0000","last_became_active":"2024-03-23T07:56:17.723965+0000","last_became_peered":"2024-03-23T07:56:17.723965+0000","last_unstale":"2024-03-23T07:56:21.992721+0000","last_undegraded":"2024-03-23T07:56:21.992721+0000","last_fullsized":"2024-03-23T07:56:21.992721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:21.059801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004697+0000","last_change":"2024-03-23T07:56:17.225692+0000","last_active":"2024-03-23T07:56:22.004697+0000","last_peered":"2024-03-23T07:56:22.004697+0000","last_clean":"2024-03-23T07:56:22.004697+0000","last_became_active":"2024-03-23T07:56:17.225384+0000","last_became_peered":"2024-03-23T07:56:17.225384+0000","last_unstale":"2024-03-23T07:56:22.004697+0000","last_undegraded":"2024-03-23T07:56:22.004697+0000","last_fullsized":"2024-03-23T07:56:22.004697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:21.655121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093432+0000","last_change":"2024-03-23T07:56:22.007586+0000","last_active":"2024-03-23T07:56:22.093432+0000","last_peered":"2024-03-23T07:56:22.093432+0000","last_clean":"2024-03-23T07:56:22.093432+0000","last_became_active":"2024-03-23T07:56:22.007159+0000","last_became_peered":"2024-03-23T07:56:22.007159+0000","last_unstale":"2024-03-23T07:56:22.093432+0000","last_undegraded":"2024-03-23T07:56:22.093432+0000","last_fullsized":"2024-03-23T07:56:22.093432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.995105+0000","last_change":"2024-03-23T07:56:17.722742+0000","last_active":"2024-03-23T07:56:21.995105+0000","last_peered":"2024-03-23T07:56:21.995105+0000","last_clean":"2024-03-23T07:56:21.995105+0000","last_became_active":"2024-03-23T07:56:17.722246+0000","last_became_peered":"2024-03-23T07:56:17.722246+0000","last_unstale":"2024-03-23T07:56:21.995105+0000","last_undegraded":"2024-03-23T07:56:21.995105+0000","last_fullsized":"2024-03-23T07:56:21.995105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994508+0000","last_change":"2024-03-23T07:56:17.212269+0000","last_active":"2024-03-23T07:56:21.994508+0000","last_peered":"2024-03-23T07:56:21.994508+0000","last_clean":"2024-03-23T07:56:21.994508+0000","last_became_active":"2024-03-23T07:56:17.211965+0000","last_became_peered":"2024-03-23T07:56:17.211965+0000","last_unstale":"2024-03-23T07:56:21.994508+0000","last_undegraded":"2024-03-23T07:56:21.994508+0000","last_fullsized":"2024-03-23T07:56:21.994508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015727+0000","last_change":"2024-03-23T07:56:22.009793+0000","last_active":"2024-03-23T07:56:22.015727+0000","last_peered":"2024-03-23T07:56:22.015727+0000","last_clean":"2024-03-23T07:56:22.015727+0000","last_became_active":"2024-03-23T07:56:22.009417+0000","last_became_peered":"2024-03-23T07:56:22.009417+0000","last_unstale":"2024-03-23T07:56:22.015727+0000","last_undegraded":"2024-03-23T07:56:22.015727+0000","last_fullsized":"2024-03-23T07:56:22.015727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003091+0000","last_change":"2024-03-23T07:56:17.726582+0000","last_active":"2024-03-23T07:56:22.003091+0000","last_peered":"2024-03-23T07:56:22.003091+0000","last_clean":"2024-03-23T07:56:22.003091+0000","last_became_active":"2024-03-23T07:56:17.726338+0000","last_became_peered":"2024-03-23T07:56:17.726338+0000","last_unstale":"2024-03-23T07:56:22.003091+0000","last_undegraded":"2024-03-23T07:56:22.003091+0000","last_fullsized":"2024-03-23T07:56:22.003091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004594+0000","last_change":"2024-03-23T07:56:17.215985+0000","last_active":"2024-03-23T07:56:22.004594+0000","last_peered":"2024-03-23T07:56:22.004594+0000","last_clean":"2024-03-23T07:56:22.004594+0000","last_became_active":"2024-03-23T07:56:17.215709+0000","last_became_peered":"2024-03-23T07:56:17.215709+0000","last_unstale":"2024-03-23T07:56:22.004594+0000","last_undegraded":"2024-03-23T07:56:22.004594+0000","last_fullsized":"2024-03-23T07:56:22.004594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:15.555700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028361+0000","last_change":"2024-03-23T07:56:22.026672+0000","last_active":"2024-03-23T07:56:22.028361+0000","last_peered":"2024-03-23T07:56:22.028361+0000","last_clean":"2024-03-23T07:56:22.028361+0000","last_became_active":"2024-03-23T07:56:22.025340+0000","last_became_peered":"2024-03-23T07:56:22.025340+0000","last_unstale":"2024-03-23T07:56:22.028361+0000","last_undegraded":"2024-03-23T07:56:22.028361+0000","last_fullsized":"2024-03-23T07:56:22.028361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:59.226182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093663+0000","last_change":"2024-03-23T07:56:17.723057+0000","last_active":"2024-03-23T07:56:22.093663+0000","last_peered":"2024-03-23T07:56:22.093663+0000","last_clean":"2024-03-23T07:56:22.093663+0000","last_became_active":"2024-03-23T07:56:17.722787+0000","last_became_peered":"2024-03-23T07:56:17.722787+0000","last_unstale":"2024-03-23T07:56:22.093663+0000","last_undegraded":"2024-03-23T07:56:22.093663+0000","last_fullsized":"2024-03-23T07:56:22.093663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.992051+0000","last_change":"2024-03-23T07:56:17.217582+0000","last_active":"2024-03-23T07:56:21.992051+0000","last_peered":"2024-03-23T07:56:21.992051+0000","last_clean":"2024-03-23T07:56:21.992051+0000","last_became_active":"2024-03-23T07:56:17.217205+0000","last_became_peered":"2024-03-23T07:56:17.217205+0000","last_unstale":"2024-03-23T07:56:21.992051+0000","last_undegraded":"2024-03-23T07:56:21.992051+0000","last_fullsized":"2024-03-23T07:56:21.992051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:23:32.182551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.027442+0000","last_change":"2024-03-23T07:56:22.023281+0000","last_active":"2024-03-23T07:56:22.027442+0000","last_peered":"2024-03-23T07:56:22.027442+0000","last_clean":"2024-03-23T07:56:22.027442+0000","last_became_active":"2024-03-23T07:56:22.022756+0000","last_became_peered":"2024-03-23T07:56:22.022756+0000","last_unstale":"2024-03-23T07:56:22.027442+0000","last_undegraded":"2024-03-23T07:56:22.027442+0000","last_fullsized":"2024-03-23T07:56:22.027442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096425+0000","last_change":"2024-03-23T07:56:17.740008+0000","last_active":"2024-03-23T07:56:22.096425+0000","last_peered":"2024-03-23T07:56:22.096425+0000","last_clean":"2024-03-23T07:56:22.096425+0000","last_became_active":"2024-03-23T07:56:17.739785+0000","last_became_peered":"2024-03-23T07:56:17.739785+0000","last_unstale":"2024-03-23T07:56:22.096425+0000","last_undegraded":"2024-03-23T07:56:22.096425+0000","last_fullsized":"2024-03-23T07:56:22.096425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:41.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003892+0000","last_change":"2024-03-23T07:56:17.221620+0000","last_active":"2024-03-23T07:56:22.003892+0000","last_peered":"2024-03-23T07:56:22.003892+0000","last_clean":"2024-03-23T07:56:22.003892+0000","last_became_active":"2024-03-23T07:56:17.221312+0000","last_became_peered":"2024-03-23T07:56:17.221312+0000","last_unstale":"2024-03-23T07:56:22.003892+0000","last_undegraded":"2024-03-23T07:56:22.003892+0000","last_fullsized":"2024-03-23T07:56:22.003892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:47.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.024787+0000","last_change":"2024-03-23T07:56:22.018139+0000","last_active":"2024-03-23T07:56:22.024787+0000","last_peered":"2024-03-23T07:56:22.024787+0000","last_clean":"2024-03-23T07:56:22.024787+0000","last_became_active":"2024-03-23T07:56:22.017722+0000","last_became_peered":"2024-03-23T07:56:22.017722+0000","last_unstale":"2024-03-23T07:56:22.024787+0000","last_undegraded":"2024-03-23T07:56:22.024787+0000","last_fullsized":"2024-03-23T07:56:22.024787+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983276+0000","last_change":"2024-03-23T07:56:17.729485+0000","last_active":"2024-03-23T07:56:21.983276+0000","last_peered":"2024-03-23T07:56:21.983276+0000","last_clean":"2024-03-23T07:56:21.983276+0000","last_became_active":"2024-03-23T07:56:17.729199+0000","last_became_peered":"2024-03-23T07:56:17.729199+0000","last_unstale":"2024-03-23T07:56:21.983276+0000","last_undegraded":"2024-03-23T07:56:21.983276+0000","last_fullsized":"2024-03-23T07:56:21.983276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:30:59.262864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014955+0000","last_change":"2024-03-23T07:56:17.222373+0000","last_active":"2024-03-23T07:56:22.014955+0000","last_peered":"2024-03-23T07:56:22.014955+0000","last_clean":"2024-03-23T07:56:22.014955+0000","last_became_active":"2024-03-23T07:56:17.222028+0000","last_became_peered":"2024-03-23T07:56:17.222028+0000","last_unstale":"2024-03-23T07:56:22.014955+0000","last_undegraded":"2024-03-23T07:56:22.014955+0000","last_fullsized":"2024-03-23T07:56:22.014955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.023027+0000","last_change":"2024-03-23T07:56:22.018041+0000","last_active":"2024-03-23T07:56:22.023027+0000","last_peered":"2024-03-23T07:56:22.023027+0000","last_clean":"2024-03-23T07:56:22.023027+0000","last_became_active":"2024-03-23T07:56:22.017588+0000","last_became_peered":"2024-03-23T07:56:22.017588+0000","last_unstale":"2024-03-23T07:56:22.023027+0000","last_undegraded":"2024-03-23T07:56:22.023027+0000","last_fullsized":"2024-03-23T07:56:22.023027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:01:30.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001772+0000","last_change":"2024-03-23T07:56:17.724698+0000","last_active":"2024-03-23T07:56:22.001772+0000","last_peered":"2024-03-23T07:56:22.001772+0000","last_clean":"2024-03-23T07:56:22.001772+0000","last_became_active":"2024-03-23T07:56:17.724382+0000","last_became_peered":"2024-03-23T07:56:17.724382+0000","last_unstale":"2024-03-23T07:56:22.001772+0000","last_undegraded":"2024-03-23T07:56:22.001772+0000","last_fullsized":"2024-03-23T07:56:22.001772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002505+0000","last_change":"2024-03-23T07:56:17.223691+0000","last_active":"2024-03-23T07:56:22.002505+0000","last_peered":"2024-03-23T07:56:22.002505+0000","last_clean":"2024-03-23T07:56:22.002505+0000","last_became_active":"2024-03-23T07:56:17.222061+0000","last_became_peered":"2024-03-23T07:56:17.222061+0000","last_unstale":"2024-03-23T07:56:22.002505+0000","last_undegraded":"2024-03-23T07:56:22.002505+0000","last_fullsized":"2024-03-23T07:56:22.002505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:00.798128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471272+0000","last_change":"2024-03-23T07:56:22.011743+0000","last_active":"2024-03-23T07:56:22.471272+0000","last_peered":"2024-03-23T07:56:22.471272+0000","last_clean":"2024-03-23T07:56:22.471272+0000","last_became_active":"2024-03-23T07:56:22.011280+0000","last_became_peered":"2024-03-23T07:56:22.011280+0000","last_unstale":"2024-03-23T07:56:22.471272+0000","last_undegraded":"2024-03-23T07:56:22.471272+0000","last_fullsized":"2024-03-23T07:56:22.471272+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:09:05.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988647+0000","last_change":"2024-03-23T07:56:17.722136+0000","last_active":"2024-03-23T07:56:21.988647+0000","last_peered":"2024-03-23T07:56:21.988647+0000","last_clean":"2024-03-23T07:56:21.988647+0000","last_became_active":"2024-03-23T07:56:17.720479+0000","last_became_peered":"2024-03-23T07:56:17.720479+0000","last_unstale":"2024-03-23T07:56:21.988647+0000","last_undegraded":"2024-03-23T07:56:21.988647+0000","last_fullsized":"2024-03-23T07:56:21.988647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003825+0000","last_change":"2024-03-23T07:56:17.227266+0000","last_active":"2024-03-23T07:56:22.003825+0000","last_peered":"2024-03-23T07:56:22.003825+0000","last_clean":"2024-03-23T07:56:22.003825+0000","last_became_active":"2024-03-23T07:56:17.226775+0000","last_became_peered":"2024-03-23T07:56:17.226775+0000","last_unstale":"2024-03-23T07:56:22.003825+0000","last_undegraded":"2024-03-23T07:56:22.003825+0000","last_fullsized":"2024-03-23T07:56:22.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005759+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:56:22.005759+0000","last_peered":"2024-03-23T07:56:22.005759+0000","last_clean":"2024-03-23T07:56:22.005759+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:56:22.005759+0000","last_undegraded":"2024-03-23T07:56:22.005759+0000","last_fullsized":"2024-03-23T07:56:22.005759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097280+0000","last_change":"2024-03-23T07:56:22.024202+0000","last_active":"2024-03-23T07:56:22.097280+0000","last_peered":"2024-03-23T07:56:22.097280+0000","last_clean":"2024-03-23T07:56:22.097280+0000","last_became_active":"2024-03-23T07:56:22.018941+0000","last_became_peered":"2024-03-23T07:56:22.018941+0000","last_unstale":"2024-03-23T07:56:22.097280+0000","last_undegraded":"2024-03-23T07:56:22.097280+0000","last_fullsized":"2024-03-23T07:56:22.097280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:04:25.917706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097267+0000","last_change":"2024-03-23T07:56:17.734848+0000","last_active":"2024-03-23T07:56:22.097267+0000","last_peered":"2024-03-23T07:56:22.097267+0000","last_clean":"2024-03-23T07:56:22.097267+0000","last_became_active":"2024-03-23T07:56:17.733233+0000","last_became_peered":"2024-03-23T07:56:17.733233+0000","last_unstale":"2024-03-23T07:56:22.097267+0000","last_undegraded":"2024-03-23T07:56:22.097267+0000","last_fullsized":"2024-03-23T07:56:22.097267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003338+0000","last_change":"2024-03-23T07:56:17.215068+0000","last_active":"2024-03-23T07:56:22.003338+0000","last_peered":"2024-03-23T07:56:22.003338+0000","last_clean":"2024-03-23T07:56:22.003338+0000","last_became_active":"2024-03-23T07:56:17.214726+0000","last_became_peered":"2024-03-23T07:56:17.214726+0000","last_unstale":"2024-03-23T07:56:22.003338+0000","last_undegraded":"2024-03-23T07:56:22.003338+0000","last_fullsized":"2024-03-23T07:56:22.003338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093494+0000","last_change":"2024-03-23T07:56:22.023739+0000","last_active":"2024-03-23T07:56:22.093494+0000","last_peered":"2024-03-23T07:56:22.093494+0000","last_clean":"2024-03-23T07:56:22.093494+0000","last_became_active":"2024-03-23T07:56:22.023366+0000","last_became_peered":"2024-03-23T07:56:22.023366+0000","last_unstale":"2024-03-23T07:56:22.093494+0000","last_undegraded":"2024-03-23T07:56:22.093494+0000","last_fullsized":"2024-03-23T07:56:22.093494+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:13.143472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096511+0000","last_change":"2024-03-23T07:56:17.737947+0000","last_active":"2024-03-23T07:56:22.096511+0000","last_peered":"2024-03-23T07:56:22.096511+0000","last_clean":"2024-03-23T07:56:22.096511+0000","last_became_active":"2024-03-23T07:56:17.733346+0000","last_became_peered":"2024-03-23T07:56:17.733346+0000","last_unstale":"2024-03-23T07:56:22.096511+0000","last_undegraded":"2024-03-23T07:56:22.096511+0000","last_fullsized":"2024-03-23T07:56:22.096511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:40:33.911397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.998748+0000","last_change":"2024-03-23T07:56:17.221283+0000","last_active":"2024-03-23T07:56:21.998748+0000","last_peered":"2024-03-23T07:56:21.998748+0000","last_clean":"2024-03-23T07:56:21.998748+0000","last_became_active":"2024-03-23T07:56:17.219961+0000","last_became_peered":"2024-03-23T07:56:17.219961+0000","last_unstale":"2024-03-23T07:56:21.998748+0000","last_undegraded":"2024-03-23T07:56:21.998748+0000","last_fullsized":"2024-03-23T07:56:21.998748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:04:11.860279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.470374+0000","last_change":"2024-03-23T07:56:22.016551+0000","last_active":"2024-03-23T07:56:22.470374+0000","last_peered":"2024-03-23T07:56:22.470374+0000","last_clean":"2024-03-23T07:56:22.470374+0000","last_became_active":"2024-03-23T07:56:22.016136+0000","last_became_peered":"2024-03-23T07:56:22.016136+0000","last_unstale":"2024-03-23T07:56:22.470374+0000","last_undegraded":"2024-03-23T07:56:22.470374+0000","last_fullsized":"2024-03-23T07:56:22.470374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:36.873288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096655+0000","last_change":"2024-03-23T07:56:17.743611+0000","last_active":"2024-03-23T07:56:22.096655+0000","last_peered":"2024-03-23T07:56:22.096655+0000","last_clean":"2024-03-23T07:56:22.096655+0000","last_became_active":"2024-03-23T07:56:17.742926+0000","last_became_peered":"2024-03-23T07:56:17.742926+0000","last_unstale":"2024-03-23T07:56:22.096655+0000","last_undegraded":"2024-03-23T07:56:22.096655+0000","last_fullsized":"2024-03-23T07:56:22.096655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.994328+0000","last_change":"2024-03-23T07:56:17.219709+0000","last_active":"2024-03-23T07:56:21.994328+0000","last_peered":"2024-03-23T07:56:21.994328+0000","last_clean":"2024-03-23T07:56:21.994328+0000","last_became_active":"2024-03-23T07:56:17.218465+0000","last_became_peered":"2024-03-23T07:56:17.218465+0000","last_unstale":"2024-03-23T07:56:21.994328+0000","last_undegraded":"2024-03-23T07:56:21.994328+0000","last_fullsized":"2024-03-23T07:56:21.994328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096964+0000","last_change":"2024-03-23T07:56:22.017346+0000","last_active":"2024-03-23T07:56:22.096964+0000","last_peered":"2024-03-23T07:56:22.096964+0000","last_clean":"2024-03-23T07:56:22.096964+0000","last_became_active":"2024-03-23T07:56:22.016949+0000","last_became_peered":"2024-03-23T07:56:22.016949+0000","last_unstale":"2024-03-23T07:56:22.096964+0000","last_undegraded":"2024-03-23T07:56:22.096964+0000","last_fullsized":"2024-03-23T07:56:22.096964+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:45.299178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093733+0000","last_change":"2024-03-23T07:56:17.742996+0000","last_active":"2024-03-23T07:56:22.093733+0000","last_peered":"2024-03-23T07:56:22.093733+0000","last_clean":"2024-03-23T07:56:22.093733+0000","last_became_active":"2024-03-23T07:56:17.742660+0000","last_became_peered":"2024-03-23T07:56:17.742660+0000","last_unstale":"2024-03-23T07:56:22.093733+0000","last_undegraded":"2024-03-23T07:56:22.093733+0000","last_fullsized":"2024-03-23T07:56:22.093733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.004353+0000","last_change":"2024-03-23T07:56:17.215901+0000","last_active":"2024-03-23T07:56:22.004353+0000","last_peered":"2024-03-23T07:56:22.004353+0000","last_clean":"2024-03-23T07:56:22.004353+0000","last_became_active":"2024-03-23T07:56:17.215638+0000","last_became_peered":"2024-03-23T07:56:17.215638+0000","last_unstale":"2024-03-23T07:56:22.004353+0000","last_undegraded":"2024-03-23T07:56:22.004353+0000","last_fullsized":"2024-03-23T07:56:22.004353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:52:29.542972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093529+0000","last_change":"2024-03-23T07:56:22.010512+0000","last_active":"2024-03-23T07:56:22.093529+0000","last_peered":"2024-03-23T07:56:22.093529+0000","last_clean":"2024-03-23T07:56:22.093529+0000","last_became_active":"2024-03-23T07:56:22.009577+0000","last_became_peered":"2024-03-23T07:56:22.009577+0000","last_unstale":"2024-03-23T07:56:22.093529+0000","last_undegraded":"2024-03-23T07:56:22.093529+0000","last_fullsized":"2024-03-23T07:56:22.093529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:35:24.844391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097072+0000","last_change":"2024-03-23T07:56:17.732277+0000","last_active":"2024-03-23T07:56:22.097072+0000","last_peered":"2024-03-23T07:56:22.097072+0000","last_clean":"2024-03-23T07:56:22.097072+0000","last_became_active":"2024-03-23T07:56:17.731487+0000","last_became_peered":"2024-03-23T07:56:17.731487+0000","last_unstale":"2024-03-23T07:56:22.097072+0000","last_undegraded":"2024-03-23T07:56:22.097072+0000","last_fullsized":"2024-03-23T07:56:22.097072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:39.462067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985538+0000","last_change":"2024-03-23T07:56:17.211010+0000","last_active":"2024-03-23T07:56:21.985538+0000","last_peered":"2024-03-23T07:56:21.985538+0000","last_clean":"2024-03-23T07:56:21.985538+0000","last_became_active":"2024-03-23T07:56:17.210710+0000","last_became_peered":"2024-03-23T07:56:17.210710+0000","last_unstale":"2024-03-23T07:56:21.985538+0000","last_undegraded":"2024-03-23T07:56:21.985538+0000","last_fullsized":"2024-03-23T07:56:21.985538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:43:39.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471596+0000","last_change":"2024-03-23T07:56:22.019325+0000","last_active":"2024-03-23T07:56:22.471596+0000","last_peered":"2024-03-23T07:56:22.471596+0000","last_clean":"2024-03-23T07:56:22.471596+0000","last_became_active":"2024-03-23T07:56:22.016793+0000","last_became_peered":"2024-03-23T07:56:22.016793+0000","last_unstale":"2024-03-23T07:56:22.471596+0000","last_undegraded":"2024-03-23T07:56:22.471596+0000","last_fullsized":"2024-03-23T07:56:22.471596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:08:59.171123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093481+0000","last_change":"2024-03-23T07:56:17.716144+0000","last_active":"2024-03-23T07:56:22.093481+0000","last_peered":"2024-03-23T07:56:22.093481+0000","last_clean":"2024-03-23T07:56:22.093481+0000","last_became_active":"2024-03-23T07:56:17.715899+0000","last_became_peered":"2024-03-23T07:56:17.715899+0000","last_unstale":"2024-03-23T07:56:22.093481+0000","last_undegraded":"2024-03-23T07:56:22.093481+0000","last_fullsized":"2024-03-23T07:56:22.093481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005380+0000","last_change":"2024-03-23T07:56:17.226828+0000","last_active":"2024-03-23T07:56:22.005380+0000","last_peered":"2024-03-23T07:56:22.005380+0000","last_clean":"2024-03-23T07:56:22.005380+0000","last_became_active":"2024-03-23T07:56:17.226551+0000","last_became_peered":"2024-03-23T07:56:17.226551+0000","last_unstale":"2024-03-23T07:56:22.005380+0000","last_undegraded":"2024-03-23T07:56:22.005380+0000","last_fullsized":"2024-03-23T07:56:22.005380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:03.843216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.471755+0000","last_change":"2024-03-23T07:56:22.019606+0000","last_active":"2024-03-23T07:56:22.471755+0000","last_peered":"2024-03-23T07:56:22.471755+0000","last_clean":"2024-03-23T07:56:22.471755+0000","last_became_active":"2024-03-23T07:56:22.019280+0000","last_became_peered":"2024-03-23T07:56:22.019280+0000","last_unstale":"2024-03-23T07:56:22.471755+0000","last_undegraded":"2024-03-23T07:56:22.471755+0000","last_fullsized":"2024-03-23T07:56:22.471755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:06:28.706338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097162+0000","last_change":"2024-03-23T07:56:17.746537+0000","last_active":"2024-03-23T07:56:22.097162+0000","last_peered":"2024-03-23T07:56:22.097162+0000","last_clean":"2024-03-23T07:56:22.097162+0000","last_became_active":"2024-03-23T07:56:17.746054+0000","last_became_peered":"2024-03-23T07:56:17.746054+0000","last_unstale":"2024-03-23T07:56:22.097162+0000","last_undegraded":"2024-03-23T07:56:22.097162+0000","last_fullsized":"2024-03-23T07:56:22.097162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:59:09.358882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003661+0000","last_change":"2024-03-23T07:56:17.225527+0000","last_active":"2024-03-23T07:56:22.003661+0000","last_peered":"2024-03-23T07:56:22.003661+0000","last_clean":"2024-03-23T07:56:22.003661+0000","last_became_active":"2024-03-23T07:56:17.224860+0000","last_became_peered":"2024-03-23T07:56:17.224860+0000","last_unstale":"2024-03-23T07:56:22.003661+0000","last_undegraded":"2024-03-23T07:56:22.003661+0000","last_fullsized":"2024-03-23T07:56:22.003661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:26.692314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.028094+0000","last_change":"2024-03-23T07:56:22.023659+0000","last_active":"2024-03-23T07:56:22.028094+0000","last_peered":"2024-03-23T07:56:22.028094+0000","last_clean":"2024-03-23T07:56:22.028094+0000","last_became_active":"2024-03-23T07:56:22.023230+0000","last_became_peered":"2024-03-23T07:56:22.023230+0000","last_unstale":"2024-03-23T07:56:22.028094+0000","last_undegraded":"2024-03-23T07:56:22.028094+0000","last_fullsized":"2024-03-23T07:56:22.028094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:17:00.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002303+0000","last_change":"2024-03-23T07:56:17.729587+0000","last_active":"2024-03-23T07:56:22.002303+0000","last_peered":"2024-03-23T07:56:22.002303+0000","last_clean":"2024-03-23T07:56:22.002303+0000","last_became_active":"2024-03-23T07:56:17.727873+0000","last_became_peered":"2024-03-23T07:56:17.727873+0000","last_unstale":"2024-03-23T07:56:22.002303+0000","last_undegraded":"2024-03-23T07:56:22.002303+0000","last_fullsized":"2024-03-23T07:56:22.002303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017227+0000","last_change":"2024-03-23T07:56:17.215408+0000","last_active":"2024-03-23T07:56:22.017227+0000","last_peered":"2024-03-23T07:56:22.017227+0000","last_clean":"2024-03-23T07:56:22.017227+0000","last_became_active":"2024-03-23T07:56:17.215155+0000","last_became_peered":"2024-03-23T07:56:17.215155+0000","last_unstale":"2024-03-23T07:56:22.017227+0000","last_undegraded":"2024-03-23T07:56:22.017227+0000","last_fullsized":"2024-03-23T07:56:22.017227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:32:48.748092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097238+0000","last_change":"2024-03-23T07:56:22.010493+0000","last_active":"2024-03-23T07:56:22.097238+0000","last_peered":"2024-03-23T07:56:22.097238+0000","last_clean":"2024-03-23T07:56:22.097238+0000","last_became_active":"2024-03-23T07:56:22.010048+0000","last_became_peered":"2024-03-23T07:56:22.010048+0000","last_unstale":"2024-03-23T07:56:22.097238+0000","last_undegraded":"2024-03-23T07:56:22.097238+0000","last_fullsized":"2024-03-23T07:56:22.097238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:16:17.490111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096554+0000","last_change":"2024-03-23T07:56:17.735587+0000","last_active":"2024-03-23T07:56:22.096554+0000","last_peered":"2024-03-23T07:56:22.096554+0000","last_clean":"2024-03-23T07:56:22.096554+0000","last_became_active":"2024-03-23T07:56:17.735319+0000","last_became_peered":"2024-03-23T07:56:17.735319+0000","last_unstale":"2024-03-23T07:56:22.096554+0000","last_undegraded":"2024-03-23T07:56:22.096554+0000","last_fullsized":"2024-03-23T07:56:22.096554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-23T07:56:36.310952+0000","last_change":"2024-03-23T07:56:17.217671+0000","last_active":"2024-03-23T07:56:36.310952+0000","last_peered":"2024-03-23T07:56:36.310952+0000","last_clean":"2024-03-23T07:56:36.310952+0000","last_became_active":"2024-03-23T07:56:17.217332+0000","last_became_peered":"2024-03-23T07:56:17.217332+0000","last_unstale":"2024-03-23T07:56:36.310952+0000","last_undegraded":"2024-03-23T07:56:36.310952+0000","last_fullsized":"2024-03-23T07:56:36.310952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:40:30.911275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018197+0000","last_change":"2024-03-23T07:56:22.013208+0000","last_active":"2024-03-23T07:56:22.018197+0000","last_peered":"2024-03-23T07:56:22.018197+0000","last_clean":"2024-03-23T07:56:22.018197+0000","last_became_active":"2024-03-23T07:56:22.012577+0000","last_became_peered":"2024-03-23T07:56:22.012577+0000","last_unstale":"2024-03-23T07:56:22.018197+0000","last_undegraded":"2024-03-23T07:56:22.018197+0000","last_fullsized":"2024-03-23T07:56:22.018197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:58:48.739523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097206+0000","last_change":"2024-03-23T07:56:17.735251+0000","last_active":"2024-03-23T07:56:22.097206+0000","last_peered":"2024-03-23T07:56:22.097206+0000","last_clean":"2024-03-23T07:56:22.097206+0000","last_became_active":"2024-03-23T07:56:17.731173+0000","last_became_peered":"2024-03-23T07:56:17.731173+0000","last_unstale":"2024-03-23T07:56:22.097206+0000","last_undegraded":"2024-03-23T07:56:22.097206+0000","last_fullsized":"2024-03-23T07:56:22.097206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:35:54.379935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018228+0000","last_change":"2024-03-23T07:56:17.216512+0000","last_active":"2024-03-23T07:56:22.018228+0000","last_peered":"2024-03-23T07:56:22.018228+0000","last_clean":"2024-03-23T07:56:22.018228+0000","last_became_active":"2024-03-23T07:56:17.215743+0000","last_became_peered":"2024-03-23T07:56:17.215743+0000","last_unstale":"2024-03-23T07:56:22.018228+0000","last_undegraded":"2024-03-23T07:56:22.018228+0000","last_fullsized":"2024-03-23T07:56:22.018228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093611+0000","last_change":"2024-03-23T07:56:22.014702+0000","last_active":"2024-03-23T07:56:22.093611+0000","last_peered":"2024-03-23T07:56:22.093611+0000","last_clean":"2024-03-23T07:56:22.093611+0000","last_became_active":"2024-03-23T07:56:22.014290+0000","last_became_peered":"2024-03-23T07:56:22.014290+0000","last_unstale":"2024-03-23T07:56:22.093611+0000","last_undegraded":"2024-03-23T07:56:22.093611+0000","last_fullsized":"2024-03-23T07:56:22.093611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:49.933778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.005107+0000","last_change":"2024-03-23T07:56:17.725552+0000","last_active":"2024-03-23T07:56:22.005107+0000","last_peered":"2024-03-23T07:56:22.005107+0000","last_clean":"2024-03-23T07:56:22.005107+0000","last_became_active":"2024-03-23T07:56:17.725241+0000","last_became_peered":"2024-03-23T07:56:17.725241+0000","last_unstale":"2024-03-23T07:56:22.005107+0000","last_undegraded":"2024-03-23T07:56:22.005107+0000","last_fullsized":"2024-03-23T07:56:22.005107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:08:14.226813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.999373+0000","last_change":"2024-03-23T07:56:17.224220+0000","last_active":"2024-03-23T07:56:21.999373+0000","last_peered":"2024-03-23T07:56:21.999373+0000","last_clean":"2024-03-23T07:56:21.999373+0000","last_became_active":"2024-03-23T07:56:17.223476+0000","last_became_peered":"2024-03-23T07:56:17.223476+0000","last_unstale":"2024-03-23T07:56:21.999373+0000","last_undegraded":"2024-03-23T07:56:21.999373+0000","last_fullsized":"2024-03-23T07:56:21.999373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:06:47.499859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031049+0000","last_change":"2024-03-23T07:56:22.027325+0000","last_active":"2024-03-23T07:56:22.031049+0000","last_peered":"2024-03-23T07:56:22.031049+0000","last_clean":"2024-03-23T07:56:22.031049+0000","last_became_active":"2024-03-23T07:56:22.026858+0000","last_became_peered":"2024-03-23T07:56:22.026858+0000","last_unstale":"2024-03-23T07:56:22.031049+0000","last_undegraded":"2024-03-23T07:56:22.031049+0000","last_fullsized":"2024-03-23T07:56:22.031049+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:38.880449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093719+0000","last_change":"2024-03-23T07:56:17.736540+0000","last_active":"2024-03-23T07:56:22.093719+0000","last_peered":"2024-03-23T07:56:22.093719+0000","last_clean":"2024-03-23T07:56:22.093719+0000","last_became_active":"2024-03-23T07:56:17.727131+0000","last_became_peered":"2024-03-23T07:56:17.727131+0000","last_unstale":"2024-03-23T07:56:22.093719+0000","last_undegraded":"2024-03-23T07:56:22.093719+0000","last_fullsized":"2024-03-23T07:56:22.093719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.006955+0000","last_change":"2024-03-23T07:56:17.215558+0000","last_active":"2024-03-23T07:56:22.006955+0000","last_peered":"2024-03-23T07:56:22.006955+0000","last_clean":"2024-03-23T07:56:22.006955+0000","last_became_active":"2024-03-23T07:56:17.214342+0000","last_became_peered":"2024-03-23T07:56:17.214342+0000","last_unstale":"2024-03-23T07:56:22.006955+0000","last_undegraded":"2024-03-23T07:56:22.006955+0000","last_fullsized":"2024-03-23T07:56:22.006955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:10:35.905673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.010553+0000","last_change":"2024-03-23T07:56:22.008000+0000","last_active":"2024-03-23T07:56:22.010553+0000","last_peered":"2024-03-23T07:56:22.010553+0000","last_clean":"2024-03-23T07:56:22.010553+0000","last_became_active":"2024-03-23T07:56:22.006996+0000","last_became_peered":"2024-03-23T07:56:22.006996+0000","last_unstale":"2024-03-23T07:56:22.010553+0000","last_undegraded":"2024-03-23T07:56:22.010553+0000","last_fullsized":"2024-03-23T07:56:22.010553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:24:55.446700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.015561+0000","last_change":"2024-03-23T07:56:22.009204+0000","last_active":"2024-03-23T07:56:22.015561+0000","last_peered":"2024-03-23T07:56:22.015561+0000","last_clean":"2024-03-23T07:56:22.015561+0000","last_became_active":"2024-03-23T07:56:22.008744+0000","last_became_peered":"2024-03-23T07:56:22.008744+0000","last_unstale":"2024-03-23T07:56:22.015561+0000","last_undegraded":"2024-03-23T07:56:22.015561+0000","last_fullsized":"2024-03-23T07:56:22.015561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:28.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.984217+0000","last_change":"2024-03-23T07:56:17.733961+0000","last_active":"2024-03-23T07:56:21.984217+0000","last_peered":"2024-03-23T07:56:21.984217+0000","last_clean":"2024-03-23T07:56:21.984217+0000","last_became_active":"2024-03-23T07:56:17.733646+0000","last_became_peered":"2024-03-23T07:56:17.733646+0000","last_unstale":"2024-03-23T07:56:21.984217+0000","last_undegraded":"2024-03-23T07:56:21.984217+0000","last_fullsized":"2024-03-23T07:56:21.984217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:08.893450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002413+0000","last_change":"2024-03-23T07:56:17.223808+0000","last_active":"2024-03-23T07:56:22.002413+0000","last_peered":"2024-03-23T07:56:22.002413+0000","last_clean":"2024-03-23T07:56:22.002413+0000","last_became_active":"2024-03-23T07:56:17.223429+0000","last_became_peered":"2024-03-23T07:56:17.223429+0000","last_unstale":"2024-03-23T07:56:22.002413+0000","last_undegraded":"2024-03-23T07:56:22.002413+0000","last_fullsized":"2024-03-23T07:56:22.002413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:57.800903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097333+0000","last_change":"2024-03-23T07:56:22.015838+0000","last_active":"2024-03-23T07:56:22.097333+0000","last_peered":"2024-03-23T07:56:22.097333+0000","last_clean":"2024-03-23T07:56:22.097333+0000","last_became_active":"2024-03-23T07:56:22.015393+0000","last_became_peered":"2024-03-23T07:56:22.015393+0000","last_unstale":"2024-03-23T07:56:22.097333+0000","last_undegraded":"2024-03-23T07:56:22.097333+0000","last_fullsized":"2024-03-23T07:56:22.097333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:35:13.840057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.985820+0000","last_change":"2024-03-23T07:56:17.729460+0000","last_active":"2024-03-23T07:56:21.985820+0000","last_peered":"2024-03-23T07:56:21.985820+0000","last_clean":"2024-03-23T07:56:21.985820+0000","last_became_active":"2024-03-23T07:56:17.728590+0000","last_became_peered":"2024-03-23T07:56:17.728590+0000","last_unstale":"2024-03-23T07:56:21.985820+0000","last_undegraded":"2024-03-23T07:56:21.985820+0000","last_fullsized":"2024-03-23T07:56:21.985820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:32:18.762479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001880+0000","last_change":"2024-03-23T07:56:17.216006+0000","last_active":"2024-03-23T07:56:22.001880+0000","last_peered":"2024-03-23T07:56:22.001880+0000","last_clean":"2024-03-23T07:56:22.001880+0000","last_became_active":"2024-03-23T07:56:17.215734+0000","last_became_peered":"2024-03-23T07:56:17.215734+0000","last_unstale":"2024-03-23T07:56:22.001880+0000","last_undegraded":"2024-03-23T07:56:22.001880+0000","last_fullsized":"2024-03-23T07:56:22.001880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:25.371351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.003800+0000","last_change":"2024-03-23T07:56:17.225732+0000","last_active":"2024-03-23T07:56:22.003800+0000","last_peered":"2024-03-23T07:56:22.003800+0000","last_clean":"2024-03-23T07:56:22.003800+0000","last_became_active":"2024-03-23T07:56:17.225216+0000","last_became_peered":"2024-03-23T07:56:17.225216+0000","last_unstale":"2024-03-23T07:56:22.003800+0000","last_undegraded":"2024-03-23T07:56:22.003800+0000","last_fullsized":"2024-03-23T07:56:22.003800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:08.861374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982688+0000","last_change":"2024-03-23T07:56:17.750834+0000","last_active":"2024-03-23T07:56:21.982688+0000","last_peered":"2024-03-23T07:56:21.982688+0000","last_clean":"2024-03-23T07:56:21.982688+0000","last_became_active":"2024-03-23T07:56:17.750483+0000","last_became_peered":"2024-03-23T07:56:17.750483+0000","last_unstale":"2024-03-23T07:56:21.982688+0000","last_undegraded":"2024-03-23T07:56:21.982688+0000","last_fullsized":"2024-03-23T07:56:21.982688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:37.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.018371+0000","last_change":"2024-03-23T07:56:22.013558+0000","last_active":"2024-03-23T07:56:22.018371+0000","last_peered":"2024-03-23T07:56:22.018371+0000","last_clean":"2024-03-23T07:56:22.018371+0000","last_became_active":"2024-03-23T07:56:22.011671+0000","last_became_peered":"2024-03-23T07:56:22.011671+0000","last_unstale":"2024-03-23T07:56:22.018371+0000","last_undegraded":"2024-03-23T07:56:22.018371+0000","last_fullsized":"2024-03-23T07:56:22.018371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:07.197610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991122+0000","last_change":"2024-03-23T07:56:17.217610+0000","last_active":"2024-03-23T07:56:21.991122+0000","last_peered":"2024-03-23T07:56:21.991122+0000","last_clean":"2024-03-23T07:56:21.991122+0000","last_became_active":"2024-03-23T07:56:17.217342+0000","last_became_peered":"2024-03-23T07:56:17.217342+0000","last_unstale":"2024-03-23T07:56:21.991122+0000","last_undegraded":"2024-03-23T07:56:21.991122+0000","last_fullsized":"2024-03-23T07:56:21.991122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:16.162255+0000","last_clean_scrub_stamp":"2024-03-23T07:56:16.162255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:47:27.578107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093588+0000","last_change":"2024-03-23T07:56:17.719621+0000","last_active":"2024-03-23T07:56:22.093588+0000","last_peered":"2024-03-23T07:56:22.093588+0000","last_clean":"2024-03-23T07:56:22.093588+0000","last_became_active":"2024-03-23T07:56:17.719296+0000","last_became_peered":"2024-03-23T07:56:17.719296+0000","last_unstale":"2024-03-23T07:56:22.093588+0000","last_undegraded":"2024-03-23T07:56:22.093588+0000","last_fullsized":"2024-03-23T07:56:22.093588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.031093+0000","last_change":"2024-03-23T07:56:22.027765+0000","last_active":"2024-03-23T07:56:22.031093+0000","last_peered":"2024-03-23T07:56:22.031093+0000","last_clean":"2024-03-23T07:56:22.031093+0000","last_became_active":"2024-03-23T07:56:22.027420+0000","last_became_peered":"2024-03-23T07:56:22.027420+0000","last_unstale":"2024-03-23T07:56:22.031093+0000","last_undegraded":"2024-03-23T07:56:22.031093+0000","last_fullsized":"2024-03-23T07:56:22.031093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:20.957920+0000","last_clean_scrub_stamp":"2024-03-23T07:56:20.957920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:25.578708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093412+0000","last_change":"2024-03-23T07:56:17.721639+0000","last_active":"2024-03-23T07:56:22.093412+0000","last_peered":"2024-03-23T07:56:22.093412+0000","last_clean":"2024-03-23T07:56:22.093412+0000","last_became_active":"2024-03-23T07:56:17.721372+0000","last_became_peered":"2024-03-23T07:56:17.721372+0000","last_unstale":"2024-03-23T07:56:22.093412+0000","last_undegraded":"2024-03-23T07:56:22.093412+0000","last_fullsized":"2024-03-23T07:56:22.093412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:21:01.809202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.014858+0000","last_change":"2024-03-23T07:56:17.741387+0000","last_active":"2024-03-23T07:56:22.014858+0000","last_peered":"2024-03-23T07:56:22.014858+0000","last_clean":"2024-03-23T07:56:22.014858+0000","last_became_active":"2024-03-23T07:56:17.741091+0000","last_became_peered":"2024-03-23T07:56:17.741091+0000","last_unstale":"2024-03-23T07:56:22.014858+0000","last_undegraded":"2024-03-23T07:56:22.014858+0000","last_fullsized":"2024-03-23T07:56:22.014858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:20:47.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097117+0000","last_change":"2024-03-23T07:56:17.729701+0000","last_active":"2024-03-23T07:56:22.097117+0000","last_peered":"2024-03-23T07:56:22.097117+0000","last_clean":"2024-03-23T07:56:22.097117+0000","last_became_active":"2024-03-23T07:56:17.729481+0000","last_became_peered":"2024-03-23T07:56:17.729481+0000","last_unstale":"2024-03-23T07:56:22.097117+0000","last_undegraded":"2024-03-23T07:56:22.097117+0000","last_fullsized":"2024-03-23T07:56:22.097117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:51:36.505153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.990887+0000","last_change":"2024-03-23T07:56:17.724914+0000","last_active":"2024-03-23T07:56:21.990887+0000","last_peered":"2024-03-23T07:56:21.990887+0000","last_clean":"2024-03-23T07:56:21.990887+0000","last_became_active":"2024-03-23T07:56:17.724627+0000","last_became_peered":"2024-03-23T07:56:17.724627+0000","last_unstale":"2024-03-23T07:56:21.990887+0000","last_undegraded":"2024-03-23T07:56:21.990887+0000","last_fullsized":"2024-03-23T07:56:21.990887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:36.852829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002999+0000","last_change":"2024-03-23T07:56:17.745053+0000","last_active":"2024-03-23T07:56:22.002999+0000","last_peered":"2024-03-23T07:56:22.002999+0000","last_clean":"2024-03-23T07:56:22.002999+0000","last_became_active":"2024-03-23T07:56:17.744775+0000","last_became_peered":"2024-03-23T07:56:17.744775+0000","last_unstale":"2024-03-23T07:56:22.002999+0000","last_undegraded":"2024-03-23T07:56:22.002999+0000","last_fullsized":"2024-03-23T07:56:22.002999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.001976+0000","last_change":"2024-03-23T07:56:17.726849+0000","last_active":"2024-03-23T07:56:22.001976+0000","last_peered":"2024-03-23T07:56:22.001976+0000","last_clean":"2024-03-23T07:56:22.001976+0000","last_became_active":"2024-03-23T07:56:17.726339+0000","last_became_peered":"2024-03-23T07:56:17.726339+0000","last_unstale":"2024-03-23T07:56:22.001976+0000","last_undegraded":"2024-03-23T07:56:22.001976+0000","last_fullsized":"2024-03-23T07:56:22.001976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:19:42.241859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093701+0000","last_change":"2024-03-23T07:56:17.736674+0000","last_active":"2024-03-23T07:56:22.093701+0000","last_peered":"2024-03-23T07:56:22.093701+0000","last_clean":"2024-03-23T07:56:22.093701+0000","last_became_active":"2024-03-23T07:56:17.728841+0000","last_became_peered":"2024-03-23T07:56:17.728841+0000","last_unstale":"2024-03-23T07:56:22.093701+0000","last_undegraded":"2024-03-23T07:56:22.093701+0000","last_fullsized":"2024-03-23T07:56:22.093701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:58.811977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000680+0000","last_change":"2024-03-23T07:56:17.734249+0000","last_active":"2024-03-23T07:56:22.000680+0000","last_peered":"2024-03-23T07:56:22.000680+0000","last_clean":"2024-03-23T07:56:22.000680+0000","last_became_active":"2024-03-23T07:56:17.733805+0000","last_became_peered":"2024-03-23T07:56:17.733805+0000","last_unstale":"2024-03-23T07:56:22.000680+0000","last_undegraded":"2024-03-23T07:56:22.000680+0000","last_fullsized":"2024-03-23T07:56:22.000680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:58:47.287496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097031+0000","last_change":"2024-03-23T07:56:17.728413+0000","last_active":"2024-03-23T07:56:22.097031+0000","last_peered":"2024-03-23T07:56:22.097031+0000","last_clean":"2024-03-23T07:56:22.097031+0000","last_became_active":"2024-03-23T07:56:17.727006+0000","last_became_peered":"2024-03-23T07:56:17.727006+0000","last_unstale":"2024-03-23T07:56:22.097031+0000","last_undegraded":"2024-03-23T07:56:22.097031+0000","last_fullsized":"2024-03-23T07:56:22.097031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:42:54.056625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.989132+0000","last_change":"2024-03-23T07:56:17.730066+0000","last_active":"2024-03-23T07:56:21.989132+0000","last_peered":"2024-03-23T07:56:21.989132+0000","last_clean":"2024-03-23T07:56:21.989132+0000","last_became_active":"2024-03-23T07:56:17.729250+0000","last_became_peered":"2024-03-23T07:56:17.729250+0000","last_unstale":"2024-03-23T07:56:21.989132+0000","last_undegraded":"2024-03-23T07:56:21.989132+0000","last_fullsized":"2024-03-23T07:56:21.989132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:28.507099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093550+0000","last_change":"2024-03-23T07:56:17.732206+0000","last_active":"2024-03-23T07:56:22.093550+0000","last_peered":"2024-03-23T07:56:22.093550+0000","last_clean":"2024-03-23T07:56:22.093550+0000","last_became_active":"2024-03-23T07:56:17.731391+0000","last_became_peered":"2024-03-23T07:56:17.731391+0000","last_unstale":"2024-03-23T07:56:22.093550+0000","last_undegraded":"2024-03-23T07:56:22.093550+0000","last_fullsized":"2024-03-23T07:56:22.093550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.982447+0000","last_change":"2024-03-23T07:56:17.727582+0000","last_active":"2024-03-23T07:56:21.982447+0000","last_peered":"2024-03-23T07:56:21.982447+0000","last_clean":"2024-03-23T07:56:21.982447+0000","last_became_active":"2024-03-23T07:56:17.727312+0000","last_became_peered":"2024-03-23T07:56:17.727312+0000","last_unstale":"2024-03-23T07:56:21.982447+0000","last_undegraded":"2024-03-23T07:56:21.982447+0000","last_fullsized":"2024-03-23T07:56:21.982447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:13.533933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096494+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096494+0000","last_peered":"2024-03-23T07:56:22.096494+0000","last_clean":"2024-03-23T07:56:22.096494+0000","last_became_active":"2024-03-23T07:56:17.743717+0000","last_became_peered":"2024-03-23T07:56:17.743717+0000","last_unstale":"2024-03-23T07:56:22.096494+0000","last_undegraded":"2024-03-23T07:56:22.096494+0000","last_fullsized":"2024-03-23T07:56:22.096494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:31:35.095859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097297+0000","last_change":"2024-03-23T07:56:17.729192+0000","last_active":"2024-03-23T07:56:22.097297+0000","last_peered":"2024-03-23T07:56:22.097297+0000","last_clean":"2024-03-23T07:56:22.097297+0000","last_became_active":"2024-03-23T07:56:17.728922+0000","last_became_peered":"2024-03-23T07:56:17.728922+0000","last_unstale":"2024-03-23T07:56:22.097297+0000","last_undegraded":"2024-03-23T07:56:22.097297+0000","last_fullsized":"2024-03-23T07:56:22.097297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.017348+0000","last_change":"2024-03-23T07:56:17.731099+0000","last_active":"2024-03-23T07:56:22.017348+0000","last_peered":"2024-03-23T07:56:22.017348+0000","last_clean":"2024-03-23T07:56:22.017348+0000","last_became_active":"2024-03-23T07:56:17.730597+0000","last_became_peered":"2024-03-23T07:56:17.730597+0000","last_unstale":"2024-03-23T07:56:22.017348+0000","last_undegraded":"2024-03-23T07:56:22.017348+0000","last_fullsized":"2024-03-23T07:56:22.017348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:59:04.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097349+0000","last_change":"2024-03-23T07:56:17.748094+0000","last_active":"2024-03-23T07:56:22.097349+0000","last_peered":"2024-03-23T07:56:22.097349+0000","last_clean":"2024-03-23T07:56:22.097349+0000","last_became_active":"2024-03-23T07:56:17.747867+0000","last_became_peered":"2024-03-23T07:56:17.747867+0000","last_unstale":"2024-03-23T07:56:22.097349+0000","last_undegraded":"2024-03-23T07:56:22.097349+0000","last_fullsized":"2024-03-23T07:56:22.097349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:52:53.450050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.097004+0000","last_change":"2024-03-23T07:56:17.731875+0000","last_active":"2024-03-23T07:56:22.097004+0000","last_peered":"2024-03-23T07:56:22.097004+0000","last_clean":"2024-03-23T07:56:22.097004+0000","last_became_active":"2024-03-23T07:56:17.731600+0000","last_became_peered":"2024-03-23T07:56:17.731600+0000","last_unstale":"2024-03-23T07:56:22.097004+0000","last_undegraded":"2024-03-23T07:56:22.097004+0000","last_fullsized":"2024-03-23T07:56:22.097004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:34:55.441592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.988753+0000","last_change":"2024-03-23T07:56:17.724024+0000","last_active":"2024-03-23T07:56:21.988753+0000","last_peered":"2024-03-23T07:56:21.988753+0000","last_clean":"2024-03-23T07:56:21.988753+0000","last_became_active":"2024-03-23T07:56:17.723743+0000","last_became_peered":"2024-03-23T07:56:17.723743+0000","last_unstale":"2024-03-23T07:56:21.988753+0000","last_undegraded":"2024-03-23T07:56:21.988753+0000","last_fullsized":"2024-03-23T07:56:21.988753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:32:51.077761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002205+0000","last_change":"2024-03-23T07:56:17.723640+0000","last_active":"2024-03-23T07:56:22.002205+0000","last_peered":"2024-03-23T07:56:22.002205+0000","last_clean":"2024-03-23T07:56:22.002205+0000","last_became_active":"2024-03-23T07:56:17.723352+0000","last_became_peered":"2024-03-23T07:56:17.723352+0000","last_unstale":"2024-03-23T07:56:22.002205+0000","last_undegraded":"2024-03-23T07:56:22.002205+0000","last_fullsized":"2024-03-23T07:56:22.002205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:51:48.788785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.002849+0000","last_change":"2024-03-23T07:56:17.745108+0000","last_active":"2024-03-23T07:56:22.002849+0000","last_peered":"2024-03-23T07:56:22.002849+0000","last_clean":"2024-03-23T07:56:22.002849+0000","last_became_active":"2024-03-23T07:56:17.744769+0000","last_became_peered":"2024-03-23T07:56:17.744769+0000","last_unstale":"2024-03-23T07:56:22.002849+0000","last_undegraded":"2024-03-23T07:56:22.002849+0000","last_fullsized":"2024-03-23T07:56:22.002849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:45.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096478+0000","last_change":"2024-03-23T07:56:17.743989+0000","last_active":"2024-03-23T07:56:22.096478+0000","last_peered":"2024-03-23T07:56:22.096478+0000","last_clean":"2024-03-23T07:56:22.096478+0000","last_became_active":"2024-03-23T07:56:17.743697+0000","last_became_peered":"2024-03-23T07:56:17.743697+0000","last_unstale":"2024-03-23T07:56:22.096478+0000","last_undegraded":"2024-03-23T07:56:22.096478+0000","last_fullsized":"2024-03-23T07:56:22.096478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:33:48.226604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096538+0000","last_change":"2024-03-23T07:56:17.737898+0000","last_active":"2024-03-23T07:56:22.096538+0000","last_peered":"2024-03-23T07:56:22.096538+0000","last_clean":"2024-03-23T07:56:22.096538+0000","last_became_active":"2024-03-23T07:56:17.737650+0000","last_became_peered":"2024-03-23T07:56:17.737650+0000","last_unstale":"2024-03-23T07:56:22.096538+0000","last_undegraded":"2024-03-23T07:56:22.096538+0000","last_fullsized":"2024-03-23T07:56:22.096538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:21.794118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096613+0000","last_change":"2024-03-23T07:56:17.741344+0000","last_active":"2024-03-23T07:56:22.096613+0000","last_peered":"2024-03-23T07:56:22.096613+0000","last_clean":"2024-03-23T07:56:22.096613+0000","last_became_active":"2024-03-23T07:56:17.740336+0000","last_became_peered":"2024-03-23T07:56:17.740336+0000","last_unstale":"2024-03-23T07:56:22.096613+0000","last_undegraded":"2024-03-23T07:56:22.096613+0000","last_fullsized":"2024-03-23T07:56:22.096613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:18:02.975603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093627+0000","last_change":"2024-03-23T07:56:17.725091+0000","last_active":"2024-03-23T07:56:22.093627+0000","last_peered":"2024-03-23T07:56:22.093627+0000","last_clean":"2024-03-23T07:56:22.093627+0000","last_became_active":"2024-03-23T07:56:17.724794+0000","last_became_peered":"2024-03-23T07:56:17.724794+0000","last_unstale":"2024-03-23T07:56:22.093627+0000","last_undegraded":"2024-03-23T07:56:22.093627+0000","last_fullsized":"2024-03-23T07:56:22.093627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:41:27.703388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.983064+0000","last_change":"2024-03-23T07:56:17.749204+0000","last_active":"2024-03-23T07:56:21.983064+0000","last_peered":"2024-03-23T07:56:21.983064+0000","last_clean":"2024-03-23T07:56:21.983064+0000","last_became_active":"2024-03-23T07:56:17.748888+0000","last_became_peered":"2024-03-23T07:56:17.748888+0000","last_unstale":"2024-03-23T07:56:21.983064+0000","last_undegraded":"2024-03-23T07:56:21.983064+0000","last_fullsized":"2024-03-23T07:56:21.983064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:38:14.142988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096378+0000","last_change":"2024-03-23T07:56:17.744023+0000","last_active":"2024-03-23T07:56:22.096378+0000","last_peered":"2024-03-23T07:56:22.096378+0000","last_clean":"2024-03-23T07:56:22.096378+0000","last_became_active":"2024-03-23T07:56:17.743764+0000","last_became_peered":"2024-03-23T07:56:17.743764+0000","last_unstale":"2024-03-23T07:56:22.096378+0000","last_undegraded":"2024-03-23T07:56:22.096378+0000","last_fullsized":"2024-03-23T07:56:22.096378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:31:29.237295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093513+0000","last_change":"2024-03-23T07:56:17.724709+0000","last_active":"2024-03-23T07:56:22.093513+0000","last_peered":"2024-03-23T07:56:22.093513+0000","last_clean":"2024-03-23T07:56:22.093513+0000","last_became_active":"2024-03-23T07:56:17.724393+0000","last_became_peered":"2024-03-23T07:56:17.724393+0000","last_unstale":"2024-03-23T07:56:22.093513+0000","last_undegraded":"2024-03-23T07:56:22.093513+0000","last_fullsized":"2024-03-23T07:56:22.093513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:31:22.666195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096409+0000","last_change":"2024-03-23T07:56:17.742970+0000","last_active":"2024-03-23T07:56:22.096409+0000","last_peered":"2024-03-23T07:56:22.096409+0000","last_clean":"2024-03-23T07:56:22.096409+0000","last_became_active":"2024-03-23T07:56:17.742629+0000","last_became_peered":"2024-03-23T07:56:17.742629+0000","last_unstale":"2024-03-23T07:56:22.096409+0000","last_undegraded":"2024-03-23T07:56:22.096409+0000","last_fullsized":"2024-03-23T07:56:22.096409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:21:07.705635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.093633+0000","last_change":"2024-03-23T07:56:17.736446+0000","last_active":"2024-03-23T07:56:22.093633+0000","last_peered":"2024-03-23T07:56:22.093633+0000","last_clean":"2024-03-23T07:56:22.093633+0000","last_became_active":"2024-03-23T07:56:17.727700+0000","last_became_peered":"2024-03-23T07:56:17.727700+0000","last_unstale":"2024-03-23T07:56:22.093633+0000","last_undegraded":"2024-03-23T07:56:22.093633+0000","last_fullsized":"2024-03-23T07:56:22.093633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:35:09.872448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:21.991994+0000","last_change":"2024-03-23T07:56:17.732549+0000","last_active":"2024-03-23T07:56:21.991994+0000","last_peered":"2024-03-23T07:56:21.991994+0000","last_clean":"2024-03-23T07:56:21.991994+0000","last_became_active":"2024-03-23T07:56:17.732276+0000","last_became_peered":"2024-03-23T07:56:17.732276+0000","last_unstale":"2024-03-23T07:56:21.991994+0000","last_undegraded":"2024-03-23T07:56:21.991994+0000","last_fullsized":"2024-03-23T07:56:21.991994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:48.012231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.000782+0000","last_change":"2024-03-23T07:56:17.732034+0000","last_active":"2024-03-23T07:56:22.000782+0000","last_peered":"2024-03-23T07:56:22.000782+0000","last_clean":"2024-03-23T07:56:22.000782+0000","last_became_active":"2024-03-23T07:56:17.731730+0000","last_became_peered":"2024-03-23T07:56:17.731730+0000","last_unstale":"2024-03-23T07:56:22.000782+0000","last_undegraded":"2024-03-23T07:56:22.000782+0000","last_fullsized":"2024-03-23T07:56:22.000782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:59:20.863874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-23T07:56:22.096639+0000","last_change":"2024-03-23T07:56:17.746986+0000","last_active":"2024-03-23T07:56:22.096639+0000","last_peered":"2024-03-23T07:56:22.096639+0000","last_clean":"2024-03-23T07:56:22.096639+0000","last_became_active":"2024-03-23T07:56:17.746621+0000","last_became_peered":"2024-03-23T07:56:17.746621+0000","last_unstale":"2024-03-23T07:56:22.096639+0000","last_undegraded":"2024-03-23T07:56:22.096639+0000","last_fullsized":"2024-03-23T07:56:22.096639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:17.173330+0000","last_clean_scrub_stamp":"2024-03-23T07:56:17.173330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:22.719041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476747,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]}]},{"osd":2,"up_from":16,"seq":68719476747,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":107805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":16,"seq":68719476747,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":694448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476749,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.482}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":108477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6379999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:56:52.546 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-23T07:56:52.546 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T07:56:52.546 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-23T07:56:52.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T07:56:52.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.787+0000 7f25d8a00640 1 Processor -- start 2024-03-23T07:56:52.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.787+0000 7f25d8a00640 1 -- start start 2024-03-23T07:56:52.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 0x7f25d40b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d8a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d40b16a0 con 0x7f25d406d390 2024-03-23T07:56:52.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d40cbfb0 con 0x7f25d40adbc0 2024-03-23T07:56:52.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40cc290 con 0x7f25d40cbab0 2024-03-23T07:56:52.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d2800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f25d406d390 0x7f25d40b2aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59562/0 (socket says 172.21.15.105:59562) 2024-03-23T07:56:52.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d2800640 1 -- 172.21.15.105:0/3697212659 learned_addr learned my addr 172.21.15.105:0/3697212659 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:52.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25c9400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 0x7f25d40b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872092003 0 0) 0x7f25d40b16a0 con 0x7f25d406d390 2024-03-23T07:56:52.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.789+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b8003520 con 0x7f25d406d390 2024-03-23T07:56:52.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778289991 0 0) 0x7f25b8003520 con 0x7f25d406d390 2024-03-23T07:56:52.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 msgr2=0x7f25d40b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:52.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 0x7f25d40b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 >> v1:172.21.15.132:6789/0 conn(0x7f25d40adbc0 legacy=0x7f25d40b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d40b18f0 con 0x7f25d406d390 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f25cc0019f0 con 0x7f25d406d390 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 >> v1:172.21.15.105:6789/0 conn(0x7f25d406d390 legacy=0x7f25d40b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80039a0 0x7f25b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 0x7f25b80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40cc290 con 0x7f25b80047c0 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b800bb10 con 0x7f25b80039a0 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b800dee0 con 0x7f25d40cbab0 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25c9400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d1e00640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80039a0 0x7f25b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.790+0000 7f25d2800640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 0x7f25b80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.791+0000 7f25c9400640 1 -- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 msgr2=0x7f25b80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.791+0000 7f25c9400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 0x7f25b80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.791+0000 7f25c9400640 1 -- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80039a0 msgr2=0x7f25b8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.791+0000 7f25c9400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80039a0 0x7f25b8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.791+0000 7f25c9400640 1 -- 172.21.15.105:0/3697212659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8003520 con 0x7f25b80047c0 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.791+0000 7f25d2800640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40cbab0 0x7f25b80051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25c9400640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25b8004c40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009ac0 tx=0x7f25b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f645aac6568abfa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc02c6b0 con 0x7f25b80047c0 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc02c810 con 0x7f25b80047c0 2024-03-23T07:56:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d1400640 1 -- 172.21.15.105:0/3697212659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc02cba0 con 0x7f25b80047c0 2024-03-23T07:56:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d8a00640 1 -- 172.21.15.105:0/3697212659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 msgr2=0x7f25b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d8a00640 1 --2- 172.21.15.105:0/3697212659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25b8004c40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009ac0 tx=0x7f25b80057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d2800640 1 -- 172.21.15.105:0/3697212659 reap_dead start 2024-03-23T07:56:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d8a00640 1 -- 172.21.15.105:0/3697212659 shutdown_connections 2024-03-23T07:56:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.792+0000 7f25d8a00640 1 -- 172.21.15.105:0/3697212659 >> 172.21.15.105:0/3697212659 conn(0x7f25d406a3d0 msgr2=0x7f25d40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.793+0000 7f25d8a00640 1 -- 172.21.15.105:0/3697212659 shutdown_connections 2024-03-23T07:56:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.793+0000 7f25d8a00640 1 -- 172.21.15.105:0/3697212659 wait complete. 2024-03-23T07:56:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.794+0000 7f25d8a00640 1 Processor -- start 2024-03-23T07:56:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.794+0000 7f25d8a00640 1 -- start start 2024-03-23T07:56:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25d413b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40acd50 0x7f25d413d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25d4159110 0x7f25d413daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d8a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d413cc00 con 0x7f25b80047c0 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d413d560 con 0x7f25d4159110 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d413bf80 con 0x7f25d40acd50 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25d413b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25d413b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33782/0 (socket says 172.21.15.105:33782) 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.795+0000 7f25d1e00640 1 -- 172.21.15.105:0/2395715515 learned_addr learned my addr 172.21.15.105:0/2395715515 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25d1e00640 1 -- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40acd50 msgr2=0x7f25d413d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25c9400640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40acd50 0x7f25d413d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25d1e00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40acd50 0x7f25d413d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25d1e00640 1 -- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25d4159110 msgr2=0x7f25d413daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25d2800640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25d4159110 0x7f25d413daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25d1e00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25d4159110 0x7f25d413daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25d1e00640 1 -- 172.21.15.105:0/2395715515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d413c260 con 0x7f25b80047c0 2024-03-23T07:56:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25d1e00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25d413b9a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25c400e550 tx=0x7f25c400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.796+0000 7f25caa00640 1 -- 172.21.15.105:0/2395715515 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c40052e0 con 0x7f25b80047c0 2024-03-23T07:56:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.797+0000 7f25caa00640 1 -- 172.21.15.105:0/2395715515 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c4005440 con 0x7f25b80047c0 2024-03-23T07:56:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.797+0000 7f25caa00640 1 -- 172.21.15.105:0/2395715515 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4003810 con 0x7f25b80047c0 2024-03-23T07:56:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.797+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d413c540 con 0x7f25b80047c0 2024-03-23T07:56:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.797+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d4141c20 con 0x7f25b80047c0 2024-03-23T07:56:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.798+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d413ce70 con 0x7f25b80047c0 2024-03-23T07:56:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.799+0000 7f25caa00640 1 -- 172.21.15.105:0/2395715515 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f25c4003970 con 0x7f25b80047c0 2024-03-23T07:56:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.800+0000 7f25caa00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25b006ac00 0x7f25b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.800+0000 7f25caa00640 1 -- 172.21.15.105:0/2395715515 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25c40893a0 con 0x7f25b80047c0 2024-03-23T07:56:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.800+0000 7f25c9400640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25b006ac00 0x7f25b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.801+0000 7f25c9400640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25b006ac00 0x7f25b006d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009ac0 tx=0x7f25bc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:52.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:52.804+0000 7f25caa00640 1 -- 172.21.15.105:0/2395715515 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f25c4054280 con 0x7f25b80047c0 2024-03-23T07:56:53.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.040+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f25d406ac20 con 0x7f25b80047c0 2024-03-23T07:56:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.040+0000 7f25caa00640 1 -- 172.21.15.105:0/2395715515 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f25c40578a0 con 0x7f25b80047c0 2024-03-23T07:56:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:53.041 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T07:56:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.043+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25b006ac00 msgr2=0x7f25b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.043+0000 7f25d8a00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25b006ac00 0x7f25b006d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009ac0 tx=0x7f25bc030000 comp rx=0 tx=0).stop 2024-03-23T07:56:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.043+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 msgr2=0x7f25d413b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.043+0000 7f25d8a00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25d413b9a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25c400e550 tx=0x7f25c400ea10 comp rx=0 tx=0).stop 2024-03-23T07:56:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 shutdown_connections 2024-03-23T07:56:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25d4159110 0x7f25d413daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f25b006ac00 0x7f25b006d0c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25d40acd50 0x7f25d413d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 --2- 172.21.15.105:0/2395715515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b80047c0 0x7f25d413b9a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 >> 172.21.15.105:0/2395715515 conn(0x7f25d406a3d0 msgr2=0x7f25d406c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 shutdown_connections 2024-03-23T07:56:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.044+0000 7f25d8a00640 1 -- 172.21.15.105:0/2395715515 wait complete. 2024-03-23T07:56:53.061 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-23T07:56:53.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.268+0000 7f8c8fe00640 1 Processor -- start 2024-03-23T07:56:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.269+0000 7f8c8fe00640 1 -- start start 2024-03-23T07:56:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.270+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 0x7f8c9015b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.270+0000 7f8c8fe00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c900b0db0 con 0x7f8c9015ed00 2024-03-23T07:56:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.270+0000 7f8c8fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c900b05a0 con 0x7f8c900ac310 2024-03-23T07:56:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.270+0000 7f8c8fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9015bd60 con 0x7f8c90069c90 2024-03-23T07:56:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8f400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8c9015ed00 0x7f8c9015c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59564/0 (socket says 172.21.15.105:59564) 2024-03-23T07:56:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8f400640 1 -- 172.21.15.105:0/346012112 learned_addr learned my addr 172.21.15.105:0/346012112 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8ea00640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 0x7f8c9015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755227819 0 0) 0x7f8c88001490 con 0x7f8c9015ed00 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c74003520 con 0x7f8c9015ed00 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39685089 0 0) 0x7f8c74003520 con 0x7f8c9015ed00 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 msgr2=0x7f8c9015b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.271+0000 7f8c8d600640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 0x7f8c9015b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 >> v1:172.21.15.132:6789/0 conn(0x7f8c900ac310 legacy=0x7f8c9006a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c900b07f0 con 0x7f8c9015ed00 2024-03-23T07:56:53.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f8c88001b80 con 0x7f8c9015ed00 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 >> v1:172.21.15.105:6789/0 conn(0x7f8c9015ed00 legacy=0x7f8c9015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c740039a0 0x7f8c74003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c74004350 0x7f8c74004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 0x7f8c74005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9015bd60 con 0x7f8c74004350 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7400b980 con 0x7f8c740039a0 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7400bb40 con 0x7f8c90069c90 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8e000640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c74004350 0x7f8c74004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8ea00640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c740039a0 0x7f8c74003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8f400640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 0x7f8c74005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8e000640 1 -- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 msgr2=0x7f8c74005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8e000640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90069c90 0x7f8c74005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8e000640 1 -- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c740039a0 msgr2=0x7f8c74003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8e000640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c740039a0 0x7f8c74003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.272+0000 7f8c8e000640 1 -- 172.21.15.105:0/346012112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c74003520 con 0x7f8c74004350 2024-03-23T07:56:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.273+0000 7f8c8ea00640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c740039a0 0x7f8c74003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.274+0000 7f8c8e000640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c74004350 0x7f8c74004820 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c001860 tx=0x7f8c740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ba4d17eb515a00 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.274+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c033410 con 0x7f8c74004350 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.274+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c033590 con 0x7f8c74004350 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.274+0000 7f8c8d600640 1 -- 172.21.15.105:0/346012112 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c033940 con 0x7f8c74004350 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.274+0000 7f8c8fe00640 1 -- 172.21.15.105:0/346012112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c74004350 msgr2=0x7f8c74004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.274+0000 7f8c8fe00640 1 --2- 172.21.15.105:0/346012112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c74004350 0x7f8c74004820 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c001860 tx=0x7f8c740057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.274+0000 7f8c8f400640 1 -- 172.21.15.105:0/346012112 reap_dead start 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.275+0000 7f8c8fe00640 1 -- 172.21.15.105:0/346012112 shutdown_connections 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.275+0000 7f8c8fe00640 1 -- 172.21.15.105:0/346012112 >> 172.21.15.105:0/346012112 conn(0x7f8c900b4770 msgr2=0x7f8c900b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.275+0000 7f8c8fe00640 1 -- 172.21.15.105:0/346012112 shutdown_connections 2024-03-23T07:56:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.275+0000 7f8c8fe00640 1 -- 172.21.15.105:0/346012112 wait complete. 2024-03-23T07:56:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.276+0000 7f8c8fe00640 1 Processor -- start 2024-03-23T07:56:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.276+0000 7f8c8fe00640 1 -- start start 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.277+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c74004350 0x7f8c9013b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.277+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90159ef0 0x7f8c90142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.277+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c900b0f50 0x7f8c900b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.277+0000 7f8c8fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90140ce0 con 0x7f8c900b0f50 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.277+0000 7f8c8fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900b1910 con 0x7f8c74004350 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.277+0000 7f8c8fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9013b660 con 0x7f8c90159ef0 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c900b0f50 0x7f8c900b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c900b0f50 0x7f8c900b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33796/0 (socket says 172.21.15.105:33796) 2024-03-23T07:56:53.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 -- 172.21.15.105:0/3994176315 learned_addr learned my addr 172.21.15.105:0/3994176315 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:53.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 -- 172.21.15.105:0/3994176315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90159ef0 msgr2=0x7f8c90142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90159ef0 0x7f8c90142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 -- 172.21.15.105:0/3994176315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c74004350 msgr2=0x7f8c9013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c74004350 0x7f8c9013b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 -- 172.21.15.105:0/3994176315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9013b900 con 0x7f8c900b0f50 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c8f400640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c900b0f50 0x7f8c900b13d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8c9006ad40 tx=0x7f8c8800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.278+0000 7f8c87400640 1 -- 172.21.15.105:0/3994176315 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800eb90 con 0x7f8c900b0f50 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.279+0000 7f8c87400640 1 -- 172.21.15.105:0/3994176315 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8800ed10 con 0x7f8c900b0f50 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.279+0000 7f8c87400640 1 -- 172.21.15.105:0/3994176315 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88013660 con 0x7f8c900b0f50 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.279+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9013bbe0 con 0x7f8c900b0f50 2024-03-23T07:56:53.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.279+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c90140ef0 con 0x7f8c900b0f50 2024-03-23T07:56:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.280+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9013c100 con 0x7f8c900b0f50 2024-03-23T07:56:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.281+0000 7f8c87400640 1 -- 172.21.15.105:0/3994176315 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8c880137c0 con 0x7f8c900b0f50 2024-03-23T07:56:53.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.282+0000 7f8c87400640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8c7006ac00 0x7f8c7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.282+0000 7f8c87400640 1 -- 172.21.15.105:0/3994176315 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8c88089920 con 0x7f8c900b0f50 2024-03-23T07:56:53.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.282+0000 7f8c8ea00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8c7006ac00 0x7f8c7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.282+0000 7f8c8ea00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8c7006ac00 0x7f8c7006d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009930 tx=0x7f8c80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:53.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.287+0000 7f8c87400640 1 -- 172.21.15.105:0/3994176315 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8c88054770 con 0x7f8c900b0f50 2024-03-23T07:56:53.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.483+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c900b0db0 con 0x7f8c900b0f50 2024-03-23T07:56:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.484+0000 7f8c87400640 1 -- 172.21.15.105:0/3994176315 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7f8c900b0db0 con 0x7f8c900b0f50 2024-03-23T07:56:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:53.486 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8c7006ac00 msgr2=0x7f8c7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8c7006ac00 0x7f8c7006d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009930 tx=0x7f8c80008040 comp rx=0 tx=0).stop 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c900b0f50 msgr2=0x7f8c900b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c900b0f50 0x7f8c900b13d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8c9006ad40 tx=0x7f8c8800e530 comp rx=0 tx=0).stop 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 shutdown_connections 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c900b0f50 0x7f8c900b13d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8c7006ac00 0x7f8c7006d0c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c90159ef0 0x7f8c90142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 --2- 172.21.15.105:0/3994176315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c74004350 0x7f8c9013b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.487+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 >> 172.21.15.105:0/3994176315 conn(0x7f8c900b4770 msgr2=0x7f8c9015d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:53.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.488+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 shutdown_connections 2024-03-23T07:56:53.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.488+0000 7f8c8fe00640 1 -- 172.21.15.105:0/3994176315 wait complete. 2024-03-23T07:56:53.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:56:53.507 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:56:53.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.715+0000 7fbd95200640 1 Processor -- start 2024-03-23T07:56:53.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.715+0000 7fbd95200640 1 -- start start 2024-03-23T07:56:53.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.716+0000 7fbd95200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd900cb420 0x7fbd9015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.716+0000 7fbd95200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd900b12d0 0x7fbd90160f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.716+0000 7fbd95200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd90160530 con 0x7fbd9015c9e0 2024-03-23T07:56:53.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.716+0000 7fbd95200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd90160af0 con 0x7fbd900cb420 2024-03-23T07:56:53.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.716+0000 7fbd95200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd901614c0 con 0x7fbd900b12d0 2024-03-23T07:56:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.717+0000 7fbd8ea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd900cb420 0x7fbd9015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.717+0000 7fbd8ea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd900cb420 0x7fbd9015dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33810/0 (socket says 172.21.15.105:33810) 2024-03-23T07:56:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.717+0000 7fbd8ea00640 1 -- 172.21.15.105:0/2477650099 learned_addr learned my addr 172.21.15.105:0/2477650099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.717+0000 7fbd8fe00640 1 --2- 172.21.15.105:0/2477650099 >> v2:172.21.15.132:3300/0 conn(0x7fbd900b12d0 0x7fbd90160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.717+0000 7fbd8ea00640 1 -- 172.21.15.105:0/2477650099 >> v2:172.21.15.132:3300/0 conn(0x7fbd900b12d0 msgr2=0x7fbd90160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.717+0000 7fbd8ea00640 1 --2- 172.21.15.105:0/2477650099 >> v2:172.21.15.132:3300/0 conn(0x7fbd900b12d0 0x7fbd90160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.717+0000 7fbd8ea00640 1 -- 172.21.15.105:0/2477650099 >> v1:172.21.15.132:6789/0 conn(0x7fbd9015c9e0 legacy=0x7fbd900cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.718+0000 7fbd8ea00640 1 -- 172.21.15.105:0/2477650099 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9015cdc0 con 0x7fbd900cb420 2024-03-23T07:56:53.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.719+0000 7fbd8ea00640 1 --2- 172.21.15.105:0/2477650099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900cb420 0x7fbd9015dae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd90160a60 tx=0x7fbd7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5d18c54c96f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.719+0000 7fbd8e000640 1 -- 172.21.15.105:0/2477650099 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c030860 con 0x7fbd900cb420 2024-03-23T07:56:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.719+0000 7fbd8e000640 1 -- 172.21.15.105:0/2477650099 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c0309c0 con 0x7fbd900cb420 2024-03-23T07:56:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.719+0000 7fbd8e000640 1 -- 172.21.15.105:0/2477650099 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c030d50 con 0x7fbd900cb420 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 -- 172.21.15.105:0/2477650099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900cb420 msgr2=0x7fbd9015dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 --2- 172.21.15.105:0/2477650099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900cb420 0x7fbd9015dae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd90160a60 tx=0x7fbd7c02b9a0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 -- 172.21.15.105:0/2477650099 shutdown_connections 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 --2- 172.21.15.105:0/2477650099 >> v2:172.21.15.132:3300/0 conn(0x7fbd900b12d0 0x7fbd90160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 --2- 172.21.15.105:0/2477650099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900cb420 0x7fbd9015dae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 -- 172.21.15.105:0/2477650099 >> 172.21.15.105:0/2477650099 conn(0x7fbd900b01b0 msgr2=0x7fbd900b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 -- 172.21.15.105:0/2477650099 shutdown_connections 2024-03-23T07:56:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.720+0000 7fbd95200640 1 -- 172.21.15.105:0/2477650099 wait complete. 2024-03-23T07:56:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.722+0000 7fbd95200640 1 Processor -- start 2024-03-23T07:56:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.722+0000 7fbd95200640 1 -- start start 2024-03-23T07:56:53.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.723+0000 7fbd95200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900b12d0 0x7fbd90159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.723+0000 7fbd95200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd900cb420 0x7fbd90150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.723+0000 7fbd95200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd9015c9e0 0x7fbd901568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.723+0000 7fbd95200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd901598c0 con 0x7fbd900b12d0 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.723+0000 7fbd95200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd90071f50 con 0x7fbd9015c9e0 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.723+0000 7fbd95200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd90159270 con 0x7fbd900cb420 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.723+0000 7fbd8f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900b12d0 0x7fbd90159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900b12d0 0x7fbd90159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33818/0 (socket says 172.21.15.105:33818) 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 -- 172.21.15.105:0/685111031 learned_addr learned my addr 172.21.15.105:0/685111031 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8fe00640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd9015c9e0 0x7fbd901568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8ea00640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd900cb420 0x7fbd90150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 -- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd900cb420 msgr2=0x7fbd90150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd900cb420 0x7fbd90150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 -- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd9015c9e0 msgr2=0x7fbd901568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd9015c9e0 0x7fbd901568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 -- 172.21.15.105:0/685111031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0094d0 con 0x7fbd900b12d0 2024-03-23T07:56:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8ea00640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd900cb420 0x7fbd90150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.724+0000 7fbd8f400640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900b12d0 0x7fbd90159d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c400 tx=0x7fbd8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.725+0000 7fbd87400640 1 -- 172.21.15.105:0/685111031 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800052d0 con 0x7fbd900b12d0 2024-03-23T07:56:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.725+0000 7fbd87400640 1 -- 172.21.15.105:0/685111031 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80005430 con 0x7fbd900b12d0 2024-03-23T07:56:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.725+0000 7fbd87400640 1 -- 172.21.15.105:0/685111031 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80003830 con 0x7fbd900b12d0 2024-03-23T07:56:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.725+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd90159550 con 0x7fbd900b12d0 2024-03-23T07:56:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.725+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd90151aa0 con 0x7fbd900b12d0 2024-03-23T07:56:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.726+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9006ab20 con 0x7fbd900b12d0 2024-03-23T07:56:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.727+0000 7fbd87400640 1 -- 172.21.15.105:0/685111031 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbd80003990 con 0x7fbd900b12d0 2024-03-23T07:56:53.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.728+0000 7fbd87400640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbd7006ac30 0x7fbd7006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:53.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.728+0000 7fbd87400640 1 -- 172.21.15.105:0/685111031 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd80088c50 con 0x7fbd900b12d0 2024-03-23T07:56:53.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.728+0000 7fbd8ea00640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbd7006ac30 0x7fbd7006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:53.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.729+0000 7fbd8ea00640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbd7006ac30 0x7fbd7006d0f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c036000 tx=0x7fbd7c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:53.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.732+0000 7fbd87400640 1 -- 172.21.15.105:0/685111031 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbd80053aa0 con 0x7fbd900b12d0 2024-03-23T07:56:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.927+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd9004e030 con 0x7fbd900b12d0 2024-03-23T07:56:53.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.928+0000 7fbd87400640 1 -- 172.21.15.105:0/685111031 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd80017020 con 0x7fbd900b12d0 2024-03-23T07:56:53.929 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:56:53.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbd7006ac30 msgr2=0x7fbd7006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbd7006ac30 0x7fbd7006d0f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c036000 tx=0x7fbd7c03b040 comp rx=0 tx=0).stop 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900b12d0 msgr2=0x7fbd90159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900b12d0 0x7fbd90159d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c400 tx=0x7fbd8000c8c0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 shutdown_connections 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd9015c9e0 0x7fbd901568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbd7006ac30 0x7fbd7006d0f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd900cb420 0x7fbd90150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 --2- 172.21.15.105:0/685111031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd900b12d0 0x7fbd90159d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 >> 172.21.15.105:0/685111031 conn(0x7fbd900b01b0 msgr2=0x7fbd9006c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 shutdown_connections 2024-03-23T07:56:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:53.931+0000 7fbd95200640 1 -- 172.21.15.105:0/685111031 wait complete. 2024-03-23T07:56:53.951 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:56:54.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.156+0000 7f47fec00640 1 Processor -- start 2024-03-23T07:56:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.156+0000 7f47fec00640 1 -- start start 2024-03-23T07:56:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.157+0000 7f47fec00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47f80cf040 0x7f47f81604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.157+0000 7f47fec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f8160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47fec00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f806d390 con 0x7f47f81456a0 2024-03-23T07:56:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47fec00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80ad890 con 0x7f47f80cf040 2024-03-23T07:56:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47fec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8145a80 con 0x7f47f8070cd0 2024-03-23T07:56:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47efe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47f80cf040 0x7f47f81604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47fcc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f8160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47efe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47f80cf040 0x7f47f81604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:38374/0 (socket says 172.21.15.105:38374) 2024-03-23T07:56:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47efe00640 1 -- 172.21.15.105:0/2086001398 learned_addr learned my addr 172.21.15.105:0/2086001398 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.158+0000 7f47fcc00640 1 --2- 172.21.15.105:0/2086001398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f8160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:52586/0 (socket says 172.21.15.105:52586) 2024-03-23T07:56:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.159+0000 7f47fcc00640 1 -- 172.21.15.105:0/2086001398 >> v2:172.21.15.132:3300/0 conn(0x7f47f80cf040 msgr2=0x7f47f81604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.159+0000 7f47fcc00640 1 --2- 172.21.15.105:0/2086001398 >> v2:172.21.15.132:3300/0 conn(0x7f47f80cf040 0x7f47f81604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.159+0000 7f47fcc00640 1 -- 172.21.15.105:0/2086001398 >> v1:172.21.15.132:6789/0 conn(0x7f47f81456a0 legacy=0x7f47f80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.159+0000 7f47fcc00640 1 -- 172.21.15.105:0/2086001398 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f806d670 con 0x7f47f8070cd0 2024-03-23T07:56:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.160+0000 7f47fcc00640 1 --2- 172.21.15.105:0/2086001398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f8160a00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009d90 tx=0x7f47f002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2aef4d70de085ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:54.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.161+0000 7f47f7400640 1 -- 172.21.15.105:0/2086001398 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f002cc60 con 0x7f47f8070cd0 2024-03-23T07:56:54.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.161+0000 7f47f7400640 1 -- 172.21.15.105:0/2086001398 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f002cdc0 con 0x7f47f8070cd0 2024-03-23T07:56:54.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.161+0000 7f47f7400640 1 -- 172.21.15.105:0/2086001398 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0034660 con 0x7f47f8070cd0 2024-03-23T07:56:54.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 -- 172.21.15.105:0/2086001398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 msgr2=0x7f47f8160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 --2- 172.21.15.105:0/2086001398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f8160a00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009d90 tx=0x7f47f002beb0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 -- 172.21.15.105:0/2086001398 shutdown_connections 2024-03-23T07:56:54.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 --2- 172.21.15.105:0/2086001398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f8160a00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 --2- 172.21.15.105:0/2086001398 >> v2:172.21.15.132:3300/0 conn(0x7f47f80cf040 0x7f47f81604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 -- 172.21.15.105:0/2086001398 >> 172.21.15.105:0/2086001398 conn(0x7f47f806a0b0 msgr2=0x7f47f806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:54.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 -- 172.21.15.105:0/2086001398 shutdown_connections 2024-03-23T07:56:54.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.162+0000 7f47fec00640 1 -- 172.21.15.105:0/2086001398 wait complete. 2024-03-23T07:56:54.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.164+0000 7f47fec00640 1 Processor -- start 2024-03-23T07:56:54.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.164+0000 7f47fec00640 1 -- start start 2024-03-23T07:56:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47fec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f80c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47fec00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80cf040 0x7f47f81f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47fec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f81456a0 0x7f47f815d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47fec00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f813e4d0 con 0x7f47f80cf040 2024-03-23T07:56:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47fec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80c3b10 con 0x7f47f81456a0 2024-03-23T07:56:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47fec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f815e490 con 0x7f47f8070cd0 2024-03-23T07:56:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47efe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80cf040 0x7f47f81f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47efe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80cf040 0x7f47f81f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33828/0 (socket says 172.21.15.105:33828) 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47efe00640 1 -- 172.21.15.105:0/2460678001 learned_addr learned my addr 172.21.15.105:0/2460678001 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47fcc00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f81456a0 0x7f47f815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.165+0000 7f47f7e00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f80c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47efe00640 1 -- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 msgr2=0x7f47f80c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47efe00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f80c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47efe00640 1 -- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f81456a0 msgr2=0x7f47f815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47efe00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f81456a0 0x7f47f815d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47efe00640 1 -- 172.21.15.105:0/2460678001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f0009a70 con 0x7f47f80cf040 2024-03-23T07:56:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47fcc00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f81456a0 0x7f47f815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:54.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47efe00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80cf040 0x7f47f81f2780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009fd0 tx=0x7f47e000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:54.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47f4c00640 1 -- 172.21.15.105:0/2460678001 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000c940 con 0x7f47f80cf040 2024-03-23T07:56:54.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47f4c00640 1 -- 172.21.15.105:0/2460678001 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e000caa0 con 0x7f47f80cf040 2024-03-23T07:56:54.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47f4c00640 1 -- 172.21.15.105:0/2460678001 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000ce30 con 0x7f47f80cf040 2024-03-23T07:56:54.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.166+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f815e770 con 0x7f47f80cf040 2024-03-23T07:56:54.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.167+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f81f5410 con 0x7f47f80cf040 2024-03-23T07:56:54.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.168+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f815e2f0 con 0x7f47f80cf040 2024-03-23T07:56:54.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.169+0000 7f47f4c00640 1 -- 172.21.15.105:0/2460678001 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f47e0013470 con 0x7f47f80cf040 2024-03-23T07:56:54.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.170+0000 7f47f4c00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d406ac30 0x7f47d406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.170+0000 7f47f4c00640 1 -- 172.21.15.105:0/2460678001 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f47e0089760 con 0x7f47f80cf040 2024-03-23T07:56:54.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.170+0000 7f47f7e00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d406ac30 0x7f47d406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.171+0000 7f47f7e00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d406ac30 0x7f47d406d0f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47e8000f60 tx=0x7f47e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:54.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.174+0000 7f47f4c00640 1 -- 172.21.15.105:0/2460678001 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f47f815e2f0 con 0x7f47f80cf040 2024-03-23T07:56:54.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.369+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47f815e2f0 con 0x7f47f80cf040 2024-03-23T07:56:54.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.370+0000 7f47f4c00640 1 -- 172.21.15.105:0/2460678001 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47f815e2f0 con 0x7f47f80cf040 2024-03-23T07:56:54.371 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d406ac30 msgr2=0x7f47d406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d406ac30 0x7f47d406d0f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47e8000f60 tx=0x7f47e8009040 comp rx=0 tx=0).stop 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80cf040 msgr2=0x7f47f81f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80cf040 0x7f47f81f2780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009fd0 tx=0x7f47e000ed40 comp rx=0 tx=0).stop 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 shutdown_connections 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f81456a0 0x7f47f815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80cf040 0x7f47f81f2780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d406ac30 0x7f47d406d0f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 --2- 172.21.15.105:0/2460678001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f8070cd0 0x7f47f80c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 >> 172.21.15.105:0/2460678001 conn(0x7f47f806a0b0 msgr2=0x7f47f81422b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:54.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 shutdown_connections 2024-03-23T07:56:54.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.373+0000 7f47fec00640 1 -- 172.21.15.105:0/2460678001 wait complete. 2024-03-23T07:56:54.391 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:56:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.598+0000 7f34b0400640 1 Processor -- start 2024-03-23T07:56:54.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.598+0000 7f34b0400640 1 -- start start 2024-03-23T07:56:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.599+0000 7f34b0400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34a80ce930 0x7f34a80cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34b0400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a8159c10 con 0x7f34a8159830 2024-03-23T07:56:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34b0400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a80cd840 con 0x7f34a80ca7b0 2024-03-23T07:56:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34b0400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a806de50 con 0x7f34a80ce930 2024-03-23T07:56:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34ada00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f34a8159830 0x7f34a80cc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59566/0 (socket says 172.21.15.105:59566) 2024-03-23T07:56:54.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34ada00640 1 -- 172.21.15.105:0/2050964754 learned_addr learned my addr 172.21.15.105:0/2050964754 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34ae400640 1 --2- 172.21.15.105:0/2050964754 >> v2:172.21.15.132:3300/0 conn(0x7f34a80ce930 0x7f34a80cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141740203 0 0) 0x7f34a8159c10 con 0x7f34a8159830 2024-03-23T07:56:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.600+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3490003520 con 0x7f34a8159830 2024-03-23T07:56:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1261766731 0 0) 0x7f3490003520 con 0x7f34a8159830 2024-03-23T07:56:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 >> v2:172.21.15.132:3300/0 conn(0x7f34a80ce930 msgr2=0x7f34a80cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 --2- 172.21.15.105:0/2050964754 >> v2:172.21.15.132:3300/0 conn(0x7f34a80ce930 0x7f34a80cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 >> v1:172.21.15.132:6789/0 conn(0x7f34a80ca7b0 legacy=0x7f34a80cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a80ccc30 con 0x7f34a8159830 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f349c001670 con 0x7f34a8159830 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 >> v1:172.21.15.105:6789/0 conn(0x7f34a8159830 legacy=0x7f34a80cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900039f0 0x7f3490003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900047c0 0x7f3490004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3490005180 0x7f349000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349000c050 con 0x7f34900039f0 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349000c1c0 con 0x7f34900047c0 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349000e590 con 0x7f3490005180 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34ada00640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900039f0 0x7f3490003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34ad000640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900047c0 0x7f3490004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34ae400640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3490005180 0x7f349000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.601+0000 7f34ada00640 1 -- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3490005180 msgr2=0x7f349000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:54.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.602+0000 7f34ada00640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3490005180 0x7f349000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.602+0000 7f34ada00640 1 -- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900047c0 msgr2=0x7f3490004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.602+0000 7f34ada00640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900047c0 0x7f3490004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.602+0000 7f34ada00640 1 -- 172.21.15.105:0/2050964754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490003520 con 0x7f34900039f0 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.602+0000 7f34ae400640 1 -- 172.21.15.105:0/2050964754 reap_dead start 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.603+0000 7f34ada00640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900039f0 0x7f3490003e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f349c00c0e0 tx=0x7f34900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ec043bf2659d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.603+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c005ad0 con 0x7f34900039f0 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.603+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c005c30 con 0x7f34900039f0 2024-03-23T07:56:54.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.603+0000 7f34a7e00640 1 -- 172.21.15.105:0/2050964754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0072b0 con 0x7f34900039f0 2024-03-23T07:56:54.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.603+0000 7f34b0400640 1 -- 172.21.15.105:0/2050964754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900039f0 msgr2=0x7f3490003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.603+0000 7f34b0400640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900039f0 0x7f3490003e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f349c00c0e0 tx=0x7f34900057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.604+0000 7f34b0400640 1 -- 172.21.15.105:0/2050964754 shutdown_connections 2024-03-23T07:56:54.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.604+0000 7f34b0400640 1 --2- 172.21.15.105:0/2050964754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900039f0 0x7f3490003e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.604+0000 7f34b0400640 1 -- 172.21.15.105:0/2050964754 >> 172.21.15.105:0/2050964754 conn(0x7f34a806a0b0 msgr2=0x7f34a806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:54.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.604+0000 7f34b0400640 1 -- 172.21.15.105:0/2050964754 shutdown_connections 2024-03-23T07:56:54.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.604+0000 7f34b0400640 1 -- 172.21.15.105:0/2050964754 wait complete. 2024-03-23T07:56:54.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.605+0000 7f34b0400640 1 Processor -- start 2024-03-23T07:56:54.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.606+0000 7f34b0400640 1 -- start start 2024-03-23T07:56:54.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.606+0000 7f34b0400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900039f0 0x7f34a80b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.606+0000 7f34b0400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a815caf0 0x7f34a814a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34b0400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a8157c30 0x7f34a814a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34b0400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8159dc0 con 0x7f34a8157c30 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34b0400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8158130 con 0x7f34900039f0 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34b0400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a815a010 con 0x7f34a815caf0 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a8157c30 0x7f34a814a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a8157c30 0x7f34a814a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33856/0 (socket says 172.21.15.105:33856) 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 -- 172.21.15.105:0/479307896 learned_addr learned my addr 172.21.15.105:0/479307896 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ad000640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a815caf0 0x7f34a814a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ada00640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900039f0 0x7f34a80b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 -- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a815caf0 msgr2=0x7f34a814a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a815caf0 0x7f34a814a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 -- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900039f0 msgr2=0x7f34a80b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900039f0 0x7f34a80b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.607+0000 7f34ae400640 1 -- 172.21.15.105:0/479307896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a815a2f0 con 0x7f34a8157c30 2024-03-23T07:56:54.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.608+0000 7f34ae400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a8157c30 0x7f34a814a5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f34a8071360 tx=0x7f34a000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:54.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.608+0000 7f34a5600640 1 -- 172.21.15.105:0/479307896 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0016620 con 0x7f34a8157c30 2024-03-23T07:56:54.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.608+0000 7f34a5600640 1 -- 172.21.15.105:0/479307896 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a00167a0 con 0x7f34a8157c30 2024-03-23T07:56:54.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.608+0000 7f34a5600640 1 -- 172.21.15.105:0/479307896 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0016b50 con 0x7f34a8157c30 2024-03-23T07:56:54.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.608+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a815a5d0 con 0x7f34a8157c30 2024-03-23T07:56:54.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.608+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a80b0c60 con 0x7f34a8157c30 2024-03-23T07:56:54.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.609+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a80b0e80 con 0x7f34a8157c30 2024-03-23T07:56:54.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.610+0000 7f34a5600640 1 -- 172.21.15.105:0/479307896 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f34a0016cb0 con 0x7f34a8157c30 2024-03-23T07:56:54.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.611+0000 7f34a5600640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f348806ac00 0x7f348806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:54.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.611+0000 7f34a5600640 1 -- 172.21.15.105:0/479307896 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f34a008a450 con 0x7f34a8157c30 2024-03-23T07:56:54.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.611+0000 7f34ada00640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f348806ac00 0x7f348806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:54.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.612+0000 7f34ada00640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f348806ac00 0x7f348806d0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f349c034000 tx=0x7f349c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.616+0000 7f34a5600640 1 -- 172.21.15.105:0/479307896 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f34a00552a0 con 0x7f34a8157c30 2024-03-23T07:56:54.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.809+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34a8073390 con 0x7f34a8157c30 2024-03-23T07:56:54.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.810+0000 7f34a5600640 1 -- 172.21.15.105:0/479307896 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34a00588c0 con 0x7f34a8157c30 2024-03-23T07:56:54.811 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f348806ac00 msgr2=0x7f348806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f348806ac00 0x7f348806d0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f349c034000 tx=0x7f349c009030 comp rx=0 tx=0).stop 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a8157c30 msgr2=0x7f34a814a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a8157c30 0x7f34a814a5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f34a8071360 tx=0x7f34a000f520 comp rx=0 tx=0).stop 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 shutdown_connections 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34a8157c30 0x7f34a814a5a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f348806ac00 0x7f348806d0c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a815caf0 0x7f34a814a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 --2- 172.21.15.105:0/479307896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34900039f0 0x7f34a80b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 >> 172.21.15.105:0/479307896 conn(0x7f34a806a0b0 msgr2=0x7f34a80b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 shutdown_connections 2024-03-23T07:56:54.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:54.813+0000 7f34b0400640 1 -- 172.21.15.105:0/479307896 wait complete. 2024-03-23T07:56:54.831 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:56:55.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.032+0000 7fc2be800640 1 Processor -- start 2024-03-23T07:56:55.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.032+0000 7fc2be800640 1 -- start start 2024-03-23T07:56:55.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.033+0000 7fc2be800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc2b80b3bf0 0x7fc2b80b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.033+0000 7fc2be800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b80cb5d0 con 0x7fc2b8098a40 2024-03-23T07:56:55.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.033+0000 7fc2be800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b80cb8b0 con 0x7fc2b80cf120 2024-03-23T07:56:55.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.033+0000 7fc2be800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b80cbb90 con 0x7fc2b80b3bf0 2024-03-23T07:56:55.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.034+0000 7fc2b7e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc2b80b3bf0 0x7fc2b80b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.034+0000 7fc2b7400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc2b8098a40 0x7fc2b8098e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59568/0 (socket says 172.21.15.105:59568) 2024-03-23T07:56:55.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.034+0000 7fc2b7400640 1 -- 172.21.15.105:0/1971342736 learned_addr learned my addr 172.21.15.105:0/1971342736 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:55.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.034+0000 7fc2b7e00640 1 -- 172.21.15.105:0/1971342736 >> v1:172.21.15.132:6789/0 conn(0x7fc2b80cf120 legacy=0x7fc2b80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:55.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.034+0000 7fc2b7e00640 1 -- 172.21.15.105:0/1971342736 >> v1:172.21.15.105:6789/0 conn(0x7fc2b8098a40 legacy=0x7fc2b8098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.034+0000 7fc2b7e00640 1 -- 172.21.15.105:0/1971342736 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80cbe70 con 0x7fc2b80b3bf0 2024-03-23T07:56:55.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.035+0000 7fc2b7e00640 1 --2- 172.21.15.105:0/1971342736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80b4040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009d90 tx=0x7fc2b002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaf6315a869b82c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.035+0000 7fc2b6000640 1 -- 172.21.15.105:0/1971342736 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b002cc60 con 0x7fc2b80b3bf0 2024-03-23T07:56:55.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2b6000640 1 -- 172.21.15.105:0/1971342736 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b002cdc0 con 0x7fc2b80b3bf0 2024-03-23T07:56:55.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2b6000640 1 -- 172.21.15.105:0/1971342736 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0034660 con 0x7fc2b80b3bf0 2024-03-23T07:56:55.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2be800640 1 -- 172.21.15.105:0/1971342736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80b3bf0 msgr2=0x7fc2b80b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2be800640 1 --2- 172.21.15.105:0/1971342736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80b4040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009d90 tx=0x7fc2b002beb0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2be800640 1 -- 172.21.15.105:0/1971342736 shutdown_connections 2024-03-23T07:56:55.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2be800640 1 --2- 172.21.15.105:0/1971342736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80b4040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2be800640 1 -- 172.21.15.105:0/1971342736 >> 172.21.15.105:0/1971342736 conn(0x7fc2b80b2180 msgr2=0x7fc2b80b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:55.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2be800640 1 -- 172.21.15.105:0/1971342736 shutdown_connections 2024-03-23T07:56:55.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.036+0000 7fc2be800640 1 -- 172.21.15.105:0/1971342736 wait complete. 2024-03-23T07:56:55.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.038+0000 7fc2be800640 1 Processor -- start 2024-03-23T07:56:55.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.038+0000 7fc2be800640 1 -- start start 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2be800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2b8098a40 0x7fc2b8149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2be800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2be800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80cf120 0x7fc2b814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2be800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b814a9d0 con 0x7fc2b80cf120 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2be800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8149dc0 con 0x7fc2b80b3bf0 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2be800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b814a0a0 con 0x7fc2b8098a40 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2b7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80cf120 0x7fc2b814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2b7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80cf120 0x7fc2b814ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33860/0 (socket says 172.21.15.105:33860) 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.039+0000 7fc2b7e00640 1 -- 172.21.15.105:0/3990107294 learned_addr learned my addr 172.21.15.105:0/3990107294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b7400640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2b8098a40 0x7fc2b8149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b6a00640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b7e00640 1 -- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2b8098a40 msgr2=0x7fc2b8149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b7e00640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2b8098a40 0x7fc2b8149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b7e00640 1 -- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2b80b3bf0 msgr2=0x7fc2b80aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b7e00640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b7e00640 1 -- 172.21.15.105:0/3990107294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0009a70 con 0x7fc2b80cf120 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b6a00640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2b7e00640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80cf120 0x7fc2b814ae60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009d90 tx=0x7fc2b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2af400640 1 -- 172.21.15.105:0/3990107294 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0040450 con 0x7fc2b80cf120 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2af400640 1 -- 172.21.15.105:0/3990107294 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b00405d0 con 0x7fc2b80cf120 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b814a380 con 0x7fc2b80cf120 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.040+0000 7fc2af400640 1 -- 172.21.15.105:0/3990107294 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0040980 con 0x7fc2b80cf120 2024-03-23T07:56:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.041+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b81495d0 con 0x7fc2b80cf120 2024-03-23T07:56:55.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.042+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b80b1f80 con 0x7fc2b80cf120 2024-03-23T07:56:55.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.043+0000 7fc2af400640 1 -- 172.21.15.105:0/3990107294 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc2b0040ae0 con 0x7fc2b80cf120 2024-03-23T07:56:55.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.043+0000 7fc2af400640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc29406ac30 0x7fc29406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.044+0000 7fc2af400640 1 -- 172.21.15.105:0/3990107294 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2b00abba0 con 0x7fc2b80cf120 2024-03-23T07:56:55.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.044+0000 7fc2b7400640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc29406ac30 0x7fc29406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.044+0000 7fc2b7400640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc29406ac30 0x7fc29406d0f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005880 tx=0x7fc2a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.048+0000 7fc2af400640 1 -- 172.21.15.105:0/3990107294 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc2b0076b50 con 0x7fc2b80cf120 2024-03-23T07:56:55.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.242+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc2b80cfc70 con 0x7fc2b80cf120 2024-03-23T07:56:55.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.243+0000 7fc2af400640 1 -- 172.21.15.105:0/3990107294 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc2b007a170 con 0x7fc2b80cf120 2024-03-23T07:56:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc29406ac30 msgr2=0x7fc29406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc29406ac30 0x7fc29406d0f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005880 tx=0x7fc2a8009040 comp rx=0 tx=0).stop 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80cf120 msgr2=0x7fc2b814ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80cf120 0x7fc2b814ae60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009d90 tx=0x7fc2b0004a40 comp rx=0 tx=0).stop 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 shutdown_connections 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80cf120 0x7fc2b814ae60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2b80b3bf0 0x7fc2b80aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc29406ac30 0x7fc29406d0f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 --2- 172.21.15.105:0/3990107294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2b8098a40 0x7fc2b8149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 >> 172.21.15.105:0/3990107294 conn(0x7fc2b80b2180 msgr2=0x7fc2b8098e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 shutdown_connections 2024-03-23T07:56:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.246+0000 7fc2be800640 1 -- 172.21.15.105:0/3990107294 wait complete. 2024-03-23T07:56:55.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:56:55.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.466+0000 7f0289600640 1 Processor -- start 2024-03-23T07:56:55.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.466+0000 7f0289600640 1 -- start start 2024-03-23T07:56:55.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.467+0000 7f0289600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f02840adbc0 0x7f02840b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.467+0000 7f0289600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02840cbab0 0x7f02840b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.467+0000 7f0289600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02840b16a0 con 0x7f028406d390 2024-03-23T07:56:55.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.467+0000 7f0289600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02840b4700 con 0x7f02840adbc0 2024-03-23T07:56:55.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.467+0000 7f0289600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02840cbfb0 con 0x7f02840cbab0 2024-03-23T07:56:55.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.468+0000 7f0283400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f028406d390 0x7f02840b2aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59580/0 (socket says 172.21.15.105:59580) 2024-03-23T07:56:55.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.468+0000 7f0283400640 1 -- 172.21.15.105:0/1466203616 learned_addr learned my addr 172.21.15.105:0/1466203616 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:55.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.468+0000 7f0282a00640 1 --2- 172.21.15.105:0/1466203616 >> v2:172.21.15.132:3300/0 conn(0x7f02840adbc0 0x7f02840b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.468+0000 7f027a000640 1 --2- 172.21.15.105:0/1466203616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02840cbab0 0x7f02840b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.468+0000 7f0282000640 1 -- 172.21.15.105:0/1466203616 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642482969 0 0) 0x7f02840b16a0 con 0x7f028406d390 2024-03-23T07:56:55.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.469+0000 7f0282000640 1 -- 172.21.15.105:0/1466203616 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0268003520 con 0x7f028406d390 2024-03-23T07:56:55.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.469+0000 7f027a000640 1 -- 172.21.15.105:0/1466203616 >> v2:172.21.15.132:3300/0 conn(0x7f02840adbc0 msgr2=0x7f02840b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:55.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.469+0000 7f027a000640 1 --2- 172.21.15.105:0/1466203616 >> v2:172.21.15.132:3300/0 conn(0x7f02840adbc0 0x7f02840b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.469+0000 7f027a000640 1 -- 172.21.15.105:0/1466203616 >> v1:172.21.15.105:6789/0 conn(0x7f028406d390 legacy=0x7f02840b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.469+0000 7f0282000640 1 -- 172.21.15.105:0/1466203616 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688138763 0 0) 0x7f0268003520 con 0x7f028406d390 2024-03-23T07:56:55.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.469+0000 7f027a000640 1 -- 172.21.15.105:0/1466203616 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02840cc290 con 0x7f02840cbab0 2024-03-23T07:56:55.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.470+0000 7f027a000640 1 --2- 172.21.15.105:0/1466203616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02840cbab0 0x7f02840b20f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097a0 tx=0x7f026c02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f3d64b05246c466 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.470+0000 7f0282000640 1 -- 172.21.15.105:0/1466203616 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0019f0 con 0x7f02840cbab0 2024-03-23T07:56:55.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.471+0000 7f0282000640 1 -- 172.21.15.105:0/1466203616 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c02e050 con 0x7f02840cbab0 2024-03-23T07:56:55.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.471+0000 7f0282000640 1 -- 172.21.15.105:0/1466203616 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c030710 con 0x7f02840cbab0 2024-03-23T07:56:55.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.471+0000 7f0289600640 1 -- 172.21.15.105:0/1466203616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02840cbab0 msgr2=0x7f02840b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.471+0000 7f0289600640 1 --2- 172.21.15.105:0/1466203616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02840cbab0 0x7f02840b20f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097a0 tx=0x7f026c02b710 comp rx=0 tx=0).stop 2024-03-23T07:56:55.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.472+0000 7f0289600640 1 -- 172.21.15.105:0/1466203616 shutdown_connections 2024-03-23T07:56:55.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.472+0000 7f0289600640 1 --2- 172.21.15.105:0/1466203616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02840cbab0 0x7f02840b20f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.472+0000 7f0289600640 1 --2- 172.21.15.105:0/1466203616 >> v2:172.21.15.132:3300/0 conn(0x7f02840adbc0 0x7f02840b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.472+0000 7f0289600640 1 -- 172.21.15.105:0/1466203616 >> 172.21.15.105:0/1466203616 conn(0x7f028406a3d0 msgr2=0x7f02840b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:55.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.472+0000 7f0289600640 1 -- 172.21.15.105:0/1466203616 shutdown_connections 2024-03-23T07:56:55.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.472+0000 7f0289600640 1 -- 172.21.15.105:0/1466203616 wait complete. 2024-03-23T07:56:55.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.474+0000 7f0289600640 1 Processor -- start 2024-03-23T07:56:55.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.474+0000 7f0289600640 1 -- start start 2024-03-23T07:56:55.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.475+0000 7f0289600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f028406d390 0x7f0284133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.475+0000 7f0289600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02840adbc0 0x7f0284158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.475+0000 7f0289600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02840cbab0 0x7f02841f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.475+0000 7f0289600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02841343b0 con 0x7f02840cbab0 2024-03-23T07:56:55.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.475+0000 7f0289600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028415cdd0 con 0x7f02840adbc0 2024-03-23T07:56:55.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.475+0000 7f0289600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028415caf0 con 0x7f028406d390 2024-03-23T07:56:55.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02840cbab0 0x7f02841f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02840cbab0 0x7f02841f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33876/0 (socket says 172.21.15.105:33876) 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 -- 172.21.15.105:0/868414072 learned_addr learned my addr 172.21.15.105:0/868414072 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0282a00640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f028406d390 0x7f0284133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f027a000640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02840adbc0 0x7f0284158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 -- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f028406d390 msgr2=0x7f0284133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f028406d390 0x7f0284133e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 -- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02840adbc0 msgr2=0x7f0284158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02840adbc0 0x7f0284158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 -- 172.21.15.105:0/868414072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c009400 con 0x7f02840cbab0 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.476+0000 7f0283400640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02840cbab0 0x7f02841f3c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f027c00cc70 tx=0x7f027c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.477+0000 7f027b400640 1 -- 172.21.15.105:0/868414072 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00ce80 con 0x7f02840cbab0 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.477+0000 7f027b400640 1 -- 172.21.15.105:0/868414072 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c012400 con 0x7f02840cbab0 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.477+0000 7f027b400640 1 -- 172.21.15.105:0/868414072 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0127b0 con 0x7f02840cbab0 2024-03-23T07:56:55.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.477+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028415aa10 con 0x7f02840cbab0 2024-03-23T07:56:55.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.477+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028415afd0 con 0x7f02840cbab0 2024-03-23T07:56:55.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.478+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028406ac20 con 0x7f02840cbab0 2024-03-23T07:56:55.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.479+0000 7f027b400640 1 -- 172.21.15.105:0/868414072 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f027c012910 con 0x7f02840cbab0 2024-03-23T07:56:55.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.480+0000 7f027b400640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f026006ac00 0x7f026006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.480+0000 7f027b400640 1 -- 172.21.15.105:0/868414072 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f027c089e50 con 0x7f02840cbab0 2024-03-23T07:56:55.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.480+0000 7f0282a00640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f026006ac00 0x7f026006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.481+0000 7f0282a00640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f026006ac00 0x7f026006d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0274009a30 tx=0x7f0274008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.485+0000 7f027b400640 1 -- 172.21.15.105:0/868414072 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f027c054cc0 con 0x7f02840cbab0 2024-03-23T07:56:55.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.721+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02840b2600 con 0x7f02840cbab0 2024-03-23T07:56:55.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.722+0000 7f027b400640 1 -- 172.21.15.105:0/868414072 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7f027c0582e0 con 0x7f02840cbab0 2024-03-23T07:56:55.723 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:55.723 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:56:55.723 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f026006ac00 msgr2=0x7f026006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f026006ac00 0x7f026006d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0274009a30 tx=0x7f0274008040 comp rx=0 tx=0).stop 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02840cbab0 msgr2=0x7f02841f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02840cbab0 0x7f02841f3c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f027c00cc70 tx=0x7f027c00a430 comp rx=0 tx=0).stop 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 shutdown_connections 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02840cbab0 0x7f02841f3c00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02840adbc0 0x7f0284158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f026006ac00 0x7f026006d0c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 --2- 172.21.15.105:0/868414072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f028406d390 0x7f0284133e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 >> 172.21.15.105:0/868414072 conn(0x7f028406a3d0 msgr2=0x7f02841398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:55.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 shutdown_connections 2024-03-23T07:56:55.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.725+0000 7f0289600640 1 -- 172.21.15.105:0/868414072 wait complete. 2024-03-23T07:56:55.746 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.952+0000 7fd269a00640 1 Processor -- start 2024-03-23T07:56:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.952+0000 7fd269a00640 1 -- start start 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd269a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd26413faa0 0x7fd264160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd269a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2640b1db0 0x7fd26406c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd269a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2640b38a0 con 0x7fd264160800 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd269a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2640b3b80 con 0x7fd2640b1db0 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd269a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640b3e60 con 0x7fd26413faa0 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd263400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2640b1db0 0x7fd26406c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd262000640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd264160800 0x7fd2640b16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59586/0 (socket says 172.21.15.105:59586) 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd263400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2640b1db0 0x7fd26406c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33888/0 (socket says 172.21.15.105:33888) 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd262000640 1 -- 172.21.15.105:0/2605033619 learned_addr learned my addr 172.21.15.105:0/2605033619 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.954+0000 7fd262a00640 1 --2- 172.21.15.105:0/2605033619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd26413faa0 0x7fd264160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.955+0000 7fd263400640 1 -- 172.21.15.105:0/2605033619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd26413faa0 msgr2=0x7fd264160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.955+0000 7fd263400640 1 --2- 172.21.15.105:0/2605033619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd26413faa0 0x7fd264160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.955+0000 7fd263400640 1 -- 172.21.15.105:0/2605033619 >> v1:172.21.15.105:6789/0 conn(0x7fd264160800 legacy=0x7fd2640b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.955+0000 7fd263400640 1 -- 172.21.15.105:0/2605033619 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640b4140 con 0x7fd2640b1db0 2024-03-23T07:56:55.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.955+0000 7fd261600640 1 -- 172.21.15.105:0/2605033619 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773615289 0 0) 0x7fd2640b38a0 con 0x7fd264160800 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.956+0000 7fd263400640 1 --2- 172.21.15.105:0/2605033619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2640b1db0 0x7fd26406c280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009d90 tx=0x7fd25c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7df02cf44e9c847 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.956+0000 7fd261600640 1 -- 172.21.15.105:0/2605033619 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c02cc50 con 0x7fd2640b1db0 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.956+0000 7fd261600640 1 -- 172.21.15.105:0/2605033619 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c02cdb0 con 0x7fd2640b1db0 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.956+0000 7fd261600640 1 -- 172.21.15.105:0/2605033619 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c034660 con 0x7fd2640b1db0 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.956+0000 7fd269a00640 1 -- 172.21.15.105:0/2605033619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2640b1db0 msgr2=0x7fd26406c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.956+0000 7fd269a00640 1 --2- 172.21.15.105:0/2605033619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2640b1db0 0x7fd26406c280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009d90 tx=0x7fd25c02bea0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.957+0000 7fd269a00640 1 -- 172.21.15.105:0/2605033619 shutdown_connections 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.957+0000 7fd269a00640 1 --2- 172.21.15.105:0/2605033619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2640b1db0 0x7fd26406c280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.957+0000 7fd269a00640 1 --2- 172.21.15.105:0/2605033619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd26413faa0 0x7fd264160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.957+0000 7fd269a00640 1 -- 172.21.15.105:0/2605033619 >> 172.21.15.105:0/2605033619 conn(0x7fd26406a3d0 msgr2=0x7fd2640b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:55.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.957+0000 7fd269a00640 1 -- 172.21.15.105:0/2605033619 shutdown_connections 2024-03-23T07:56:55.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.957+0000 7fd269a00640 1 -- 172.21.15.105:0/2605033619 wait complete. 2024-03-23T07:56:55.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.958+0000 7fd269a00640 1 Processor -- start 2024-03-23T07:56:55.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.958+0000 7fd269a00640 1 -- start start 2024-03-23T07:56:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.959+0000 7fd269a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2640b1db0 0x7fd26406d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.959+0000 7fd269a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd26413faa0 0x7fd26413cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.959+0000 7fd269a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd264160800 0x7fd26415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.959+0000 7fd269a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264156300 con 0x7fd26413faa0 2024-03-23T07:56:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.959+0000 7fd269a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264155780 con 0x7fd2640b1db0 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.959+0000 7fd269a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd264155a60 con 0x7fd264160800 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd26413faa0 0x7fd26413cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd26413faa0 0x7fd26413cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33902/0 (socket says 172.21.15.105:33902) 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 -- 172.21.15.105:0/3129603546 learned_addr learned my addr 172.21.15.105:0/3129603546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 -- 172.21.15.105:0/3129603546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd264160800 msgr2=0x7fd26415ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd264160800 0x7fd26415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 -- 172.21.15.105:0/3129603546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2640b1db0 msgr2=0x7fd26406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2640b1db0 0x7fd26406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 -- 172.21.15.105:0/3129603546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c009a70 con 0x7fd26413faa0 2024-03-23T07:56:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd262000640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd26413faa0 0x7fd26413cd90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c410 tx=0x7fd25000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.960+0000 7fd25aa00640 1 -- 172.21.15.105:0/3129603546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500074e0 con 0x7fd26413faa0 2024-03-23T07:56:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.961+0000 7fd25aa00640 1 -- 172.21.15.105:0/3129603546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250007660 con 0x7fd26413faa0 2024-03-23T07:56:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.961+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd264155d40 con 0x7fd26413faa0 2024-03-23T07:56:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.961+0000 7fd25aa00640 1 -- 172.21.15.105:0/3129603546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250004a20 con 0x7fd26413faa0 2024-03-23T07:56:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.961+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd264156f20 con 0x7fd26413faa0 2024-03-23T07:56:55.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.962+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2640670d0 con 0x7fd26413faa0 2024-03-23T07:56:55.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.963+0000 7fd25aa00640 1 -- 172.21.15.105:0/3129603546 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd250004b80 con 0x7fd26413faa0 2024-03-23T07:56:55.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.964+0000 7fd25aa00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd24406ac00 0x7fd24406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:55.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.964+0000 7fd25aa00640 1 -- 172.21.15.105:0/3129603546 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd2500894f0 con 0x7fd26413faa0 2024-03-23T07:56:55.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.964+0000 7fd262a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd24406ac00 0x7fd24406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.965+0000 7fd262a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd24406ac00 0x7fd24406d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2640cdc00 tx=0x7fd254008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:55.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:55.968+0000 7fd25aa00640 1 -- 172.21.15.105:0/3129603546 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd250057e70 con 0x7fd26413faa0 2024-03-23T07:56:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.149+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd264073390 con 0x7fd26413faa0 2024-03-23T07:56:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.152+0000 7fd25aa00640 1 -- 172.21.15.105:0/3129603546 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7fd250057a00 con 0x7fd26413faa0 2024-03-23T07:56:56.153 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:56.153 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd24406ac00 msgr2=0x7fd24406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd24406ac00 0x7fd24406d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2640cdc00 tx=0x7fd254008040 comp rx=0 tx=0).stop 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd26413faa0 msgr2=0x7fd26413cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd26413faa0 0x7fd26413cd90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c410 tx=0x7fd25000c8d0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 shutdown_connections 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd24406ac00 0x7fd24406d0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd264160800 0x7fd26415ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd26413faa0 0x7fd26413cd90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 --2- 172.21.15.105:0/3129603546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2640b1db0 0x7fd26406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.154+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 >> 172.21.15.105:0/3129603546 conn(0x7fd26406a3d0 msgr2=0x7fd264070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.155+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 shutdown_connections 2024-03-23T07:56:56.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.155+0000 7fd269a00640 1 -- 172.21.15.105:0/3129603546 wait complete. 2024-03-23T07:56:56.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:56:56.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:56:56.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.368+0000 7f9658a00640 1 Processor -- start 2024-03-23T07:56:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.368+0000 7f9658a00640 1 -- start start 2024-03-23T07:56:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9658a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96540add30 0x7f96540ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9658a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96540cf250 con 0x7f96540ce930 2024-03-23T07:56:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9658a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96540cdba0 con 0x7f965406ce90 2024-03-23T07:56:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9658a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96540ae1c0 con 0x7f96540add30 2024-03-23T07:56:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9651e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f96540ce930 0x7f96540cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59600/0 (socket says 172.21.15.105:59600) 2024-03-23T07:56:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9651e00640 1 -- 172.21.15.105:0/1350110206 learned_addr learned my addr 172.21.15.105:0/1350110206 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9651400640 1 --2- 172.21.15.105:0/1350110206 >> v2:172.21.15.105:3300/0 conn(0x7f96540add30 0x7f96540ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9650a00640 1 -- 172.21.15.105:0/1350110206 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968714009 0 0) 0x7f96540cf250 con 0x7f96540ce930 2024-03-23T07:56:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.370+0000 7f9650a00640 1 -- 172.21.15.105:0/1350110206 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9638003520 con 0x7f96540ce930 2024-03-23T07:56:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.371+0000 7f9651400640 1 -- 172.21.15.105:0/1350110206 >> v1:172.21.15.132:6789/0 conn(0x7f965406ce90 legacy=0x7f965415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.371+0000 7f9651400640 1 -- 172.21.15.105:0/1350110206 >> v1:172.21.15.105:6789/0 conn(0x7f96540ce930 legacy=0x7f96540cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.371+0000 7f9651400640 1 -- 172.21.15.105:0/1350110206 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540cde80 con 0x7f96540add30 2024-03-23T07:56:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.372+0000 7f9651400640 1 --2- 172.21.15.105:0/1350110206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540add30 0x7f96540ced10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f96400097a0 tx=0x7f964002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4377611e1735b879 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.372+0000 7f9650a00640 1 -- 172.21.15.105:0/1350110206 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640031830 con 0x7f96540add30 2024-03-23T07:56:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.372+0000 7f9650a00640 1 -- 172.21.15.105:0/1350110206 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9640031990 con 0x7f96540add30 2024-03-23T07:56:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.372+0000 7f9650a00640 1 -- 172.21.15.105:0/1350110206 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640031d20 con 0x7f96540add30 2024-03-23T07:56:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.372+0000 7f9658a00640 1 -- 172.21.15.105:0/1350110206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540add30 msgr2=0x7f96540ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.372+0000 7f9658a00640 1 --2- 172.21.15.105:0/1350110206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540add30 0x7f96540ced10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f96400097a0 tx=0x7f964002b8d0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.373+0000 7f9658a00640 1 -- 172.21.15.105:0/1350110206 shutdown_connections 2024-03-23T07:56:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.373+0000 7f9658a00640 1 --2- 172.21.15.105:0/1350110206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540add30 0x7f96540ced10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.373+0000 7f9658a00640 1 -- 172.21.15.105:0/1350110206 >> 172.21.15.105:0/1350110206 conn(0x7f965406a0b0 msgr2=0x7f965406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.373+0000 7f9658a00640 1 -- 172.21.15.105:0/1350110206 shutdown_connections 2024-03-23T07:56:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.373+0000 7f9658a00640 1 -- 172.21.15.105:0/1350110206 wait complete. 2024-03-23T07:56:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.374+0000 7f9658a00640 1 Processor -- start 2024-03-23T07:56:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.374+0000 7f9658a00640 1 -- start start 2024-03-23T07:56:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.375+0000 7f9658a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f965406ce90 0x7f9654157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.375+0000 7f9658a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96540add30 0x7f965415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.375+0000 7f9658a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540ce930 0x7f96541f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.375+0000 7f9658a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654151100 con 0x7f96540ce930 2024-03-23T07:56:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.375+0000 7f9658a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654156f50 con 0x7f965406ce90 2024-03-23T07:56:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.375+0000 7f9658a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96541f3ed0 con 0x7f96540add30 2024-03-23T07:56:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540ce930 0x7f96541f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540ce930 0x7f96541f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33924/0 (socket says 172.21.15.105:33924) 2024-03-23T07:56:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 -- 172.21.15.105:0/1438386163 learned_addr learned my addr 172.21.15.105:0/1438386163 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 -- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96540add30 msgr2=0x7f965415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96540add30 0x7f965415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 -- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f965406ce90 msgr2=0x7f9654157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9651e00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f965406ce90 0x7f9654157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f965406ce90 0x7f9654157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9652800640 1 -- 172.21.15.105:0/1438386163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640009400 con 0x7f96540ce930 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.376+0000 7f9651e00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f965406ce90 0x7f9654157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.377+0000 7f9652800640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540ce930 0x7f96541f34f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c7a0 tx=0x7f964c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.377+0000 7f964a000640 1 -- 172.21.15.105:0/1438386163 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c003b70 con 0x7f96540ce930 2024-03-23T07:56:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.377+0000 7f964a000640 1 -- 172.21.15.105:0/1438386163 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c003cf0 con 0x7f96540ce930 2024-03-23T07:56:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.377+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96541f41b0 con 0x7f96540ce930 2024-03-23T07:56:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.377+0000 7f964a000640 1 -- 172.21.15.105:0/1438386163 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c003ea0 con 0x7f96540ce930 2024-03-23T07:56:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.377+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9654157c70 con 0x7f96540ce930 2024-03-23T07:56:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.378+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9654157450 con 0x7f96540ce930 2024-03-23T07:56:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.379+0000 7f964a000640 1 -- 172.21.15.105:0/1438386163 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f964c00aa90 con 0x7f96540ce930 2024-03-23T07:56:56.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.380+0000 7f964a000640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f963406ac00 0x7f963406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.380+0000 7f964a000640 1 -- 172.21.15.105:0/1438386163 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f964c0894c0 con 0x7f96540ce930 2024-03-23T07:56:56.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.380+0000 7f9651e00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f963406ac00 0x7f963406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.381+0000 7f9651e00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f963406ac00 0x7f963406d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96440011e0 tx=0x7f9644009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:56.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.384+0000 7f964a000640 1 -- 172.21.15.105:0/1438386163 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9654157450 con 0x7f96540ce930 2024-03-23T07:56:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.576+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9654157450 con 0x7f96540ce930 2024-03-23T07:56:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.577+0000 7f964a000640 1 -- 172.21.15.105:0/1438386163 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9654157450 con 0x7f96540ce930 2024-03-23T07:56:56.578 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f963406ac00 msgr2=0x7f963406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f963406ac00 0x7f963406d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96440011e0 tx=0x7f9644009040 comp rx=0 tx=0).stop 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540ce930 msgr2=0x7f96541f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540ce930 0x7f96541f34f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c7a0 tx=0x7f964c00cc60 comp rx=0 tx=0).stop 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 shutdown_connections 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96540ce930 0x7f96541f34f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f963406ac00 0x7f963406d0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96540add30 0x7f965415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 --2- 172.21.15.105:0/1438386163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f965406ce90 0x7f9654157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 >> 172.21.15.105:0/1438386163 conn(0x7f965406a0b0 msgr2=0x7f965415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:56.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 shutdown_connections 2024-03-23T07:56:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.580+0000 7f9658a00640 1 -- 172.21.15.105:0/1438386163 wait complete. 2024-03-23T07:56:56.597 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:56:56.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.795+0000 7fa77e400640 1 Processor -- start 2024-03-23T07:56:56.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.796+0000 7fa77e400640 1 -- start start 2024-03-23T07:56:56.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.797+0000 7fa77e400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa778159cd0 0x7fa7780ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.797+0000 7fa77e400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7780c8c90 con 0x7fa7780ad7a0 2024-03-23T07:56:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.797+0000 7fa77e400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7780cc6c0 con 0x7fa7780cc200 2024-03-23T07:56:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.797+0000 7fa77e400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7780cc9a0 con 0x7fa778159cd0 2024-03-23T07:56:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.797+0000 7fa77da00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa7780ad7a0 0x7fa77815a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59614/0 (socket says 172.21.15.105:59614) 2024-03-23T07:56:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.797+0000 7fa77da00640 1 -- 172.21.15.105:0/2202400868 learned_addr learned my addr 172.21.15.105:0/2202400868 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa77d000640 1 --2- 172.21.15.105:0/2202400868 >> v2:172.21.15.132:3300/0 conn(0x7fa778159cd0 0x7fa7780ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597868805 0 0) 0x7fa7780c8c90 con 0x7fa7780ad7a0 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa758003520 con 0x7fa7780ad7a0 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75929252 0 0) 0x7fa758003520 con 0x7fa7780ad7a0 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 >> v2:172.21.15.132:3300/0 conn(0x7fa778159cd0 msgr2=0x7fa7780ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 --2- 172.21.15.105:0/2202400868 >> v2:172.21.15.132:3300/0 conn(0x7fa778159cd0 0x7fa7780ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 >> v1:172.21.15.132:6789/0 conn(0x7fa7780cc200 legacy=0x7fa7780cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7780ccc80 con 0x7fa7780ad7a0 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fa7700019f0 con 0x7fa7780ad7a0 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 >> v1:172.21.15.105:6789/0 conn(0x7fa7780ad7a0 legacy=0x7fa77815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7580039c0 0x7fa758003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580047c0 0x7fa758004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7580051d0 0x7fa75800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780cc9a0 con 0x7fa7580039c0 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75800c100 con 0x7fa7580051d0 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75800c2c0 con 0x7fa7580047c0 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.798+0000 7fa77d000640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7580039c0 0x7fa758003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.799+0000 7fa77d000640 1 -- 172.21.15.105:0/2202400868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580047c0 msgr2=0x7fa758004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.799+0000 7fa77d000640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580047c0 0x7fa758004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.799+0000 7fa77d000640 1 -- 172.21.15.105:0/2202400868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7580051d0 msgr2=0x7fa75800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.799+0000 7fa77d000640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7580051d0 0x7fa75800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.799+0000 7fa77d000640 1 -- 172.21.15.105:0/2202400868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa758003520 con 0x7fa7580039c0 2024-03-23T07:56:56.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.799+0000 7fa77da00640 1 -- 172.21.15.105:0/2202400868 reap_dead start 2024-03-23T07:56:56.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.800+0000 7fa77d000640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7580039c0 0x7fa758003dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ac0 tx=0x7fa7580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd78923b77b1c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.800+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76002c6f0 con 0x7fa7580039c0 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.800+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76002c850 con 0x7fa7580039c0 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.800+0000 7fa777e00640 1 -- 172.21.15.105:0/2202400868 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76002cbe0 con 0x7fa7580039c0 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.800+0000 7fa77e400640 1 -- 172.21.15.105:0/2202400868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7580039c0 msgr2=0x7fa758003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.800+0000 7fa77e400640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7580039c0 0x7fa758003dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ac0 tx=0x7fa7580057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.801+0000 7fa77e400640 1 -- 172.21.15.105:0/2202400868 shutdown_connections 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.801+0000 7fa77e400640 1 --2- 172.21.15.105:0/2202400868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7580039c0 0x7fa758003dc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.801+0000 7fa77e400640 1 -- 172.21.15.105:0/2202400868 >> 172.21.15.105:0/2202400868 conn(0x7fa7780b01b0 msgr2=0x7fa7780b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.801+0000 7fa77e400640 1 -- 172.21.15.105:0/2202400868 shutdown_connections 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.801+0000 7fa77e400640 1 -- 172.21.15.105:0/2202400868 wait complete. 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.802+0000 7fa77e400640 1 Processor -- start 2024-03-23T07:56:56.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.802+0000 7fa77e400640 1 -- start start 2024-03-23T07:56:56.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.803+0000 7fa77e400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580039c0 0x7fa7781f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.803+0000 7fa77e400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa778098e80 0x7fa7780983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.803+0000 7fa77e400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa778071440 0x7fa7780718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.803+0000 7fa77e400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780ae0c0 con 0x7fa778071440 2024-03-23T07:56:56.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.803+0000 7fa77e400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781f2380 con 0x7fa778098e80 2024-03-23T07:56:56.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.803+0000 7fa77e400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa778071e30 con 0x7fa7580039c0 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa778071440 0x7fa7780718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa778071440 0x7fa7780718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33946/0 (socket says 172.21.15.105:33946) 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 -- 172.21.15.105:0/4057211511 learned_addr learned my addr 172.21.15.105:0/4057211511 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77d000640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580039c0 0x7fa7781f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa76fe00640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa778098e80 0x7fa7780983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 -- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580039c0 msgr2=0x7fa7781f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580039c0 0x7fa7781f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 -- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa778098e80 msgr2=0x7fa7780983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa778098e80 0x7fa7780983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 -- 172.21.15.105:0/4057211511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7780add50 con 0x7fa778071440 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.804+0000 7fa77da00640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa778071440 0x7fa7780718f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa77806bb20 tx=0x7fa77000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.805+0000 7fa775600640 1 -- 172.21.15.105:0/4057211511 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000ce50 con 0x7fa778071440 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.805+0000 7fa775600640 1 -- 172.21.15.105:0/4057211511 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77000a610 con 0x7fa778071440 2024-03-23T07:56:56.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.805+0000 7fa775600640 1 -- 172.21.15.105:0/4057211511 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000a9c0 con 0x7fa778071440 2024-03-23T07:56:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.805+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77809c3b0 con 0x7fa778071440 2024-03-23T07:56:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.805+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa778099380 con 0x7fa778071440 2024-03-23T07:56:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.806+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7780ae200 con 0x7fa778071440 2024-03-23T07:56:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.807+0000 7fa775600640 1 -- 172.21.15.105:0/4057211511 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa77000ab20 con 0x7fa778071440 2024-03-23T07:56:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.808+0000 7fa775600640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa75406ac30 0x7fa75406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.808+0000 7fa775600640 1 -- 172.21.15.105:0/4057211511 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa770089250 con 0x7fa778071440 2024-03-23T07:56:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.808+0000 7fa77d000640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa75406ac30 0x7fa75406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.808+0000 7fa77d000640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa75406ac30 0x7fa75406d0f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa760036000 tx=0x7fa760030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.812+0000 7fa775600640 1 -- 172.21.15.105:0/4057211511 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa7780ae200 con 0x7fa778071440 2024-03-23T07:56:56.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.993+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7780ae200 con 0x7fa778071440 2024-03-23T07:56:56.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.994+0000 7fa775600640 1 -- 172.21.15.105:0/4057211511 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7780ae200 con 0x7fa778071440 2024-03-23T07:56:56.995 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa75406ac30 msgr2=0x7fa75406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa75406ac30 0x7fa75406d0f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa760036000 tx=0x7fa760030000 comp rx=0 tx=0).stop 2024-03-23T07:56:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa778071440 msgr2=0x7fa7780718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa778071440 0x7fa7780718f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa77806bb20 tx=0x7fa77000cc00 comp rx=0 tx=0).stop 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 shutdown_connections 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa778071440 0x7fa7780718f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa778098e80 0x7fa7780983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa75406ac30 0x7fa75406d0f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 --2- 172.21.15.105:0/4057211511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7580039c0 0x7fa7781f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.996+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 >> 172.21.15.105:0/4057211511 conn(0x7fa7780b01b0 msgr2=0x7fa7780abde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.997+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 shutdown_connections 2024-03-23T07:56:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:56.997+0000 7fa77e400640 1 -- 172.21.15.105:0/4057211511 wait complete. 2024-03-23T07:56:57.014 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:56:57.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.221+0000 7fc7bbe00640 1 Processor -- start 2024-03-23T07:56:57.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.222+0000 7fc7bbe00640 1 -- start start 2024-03-23T07:56:57.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7bbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 0x7fc7bc165360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7bbe00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc15b2b0 con 0x7fc7bc143930 2024-03-23T07:56:57.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7bbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc0cc6b0 con 0x7fc7bc143460 2024-03-23T07:56:57.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7bbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0cc950 con 0x7fc7bc0cde20 2024-03-23T07:56:57.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7ba000640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc7bc143930 0x7fc7bc0cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33936/0 (socket says 172.21.15.105:33936) 2024-03-23T07:56:57.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7ba000640 1 -- 172.21.15.105:0/2088895281 learned_addr learned my addr 172.21.15.105:0/2088895281 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7bb400640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 0x7fc7bc165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.224+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913983282 0 0) 0x7fc7bc15b2b0 con 0x7fc7bc143930 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a0003520 con 0x7fc7bc143930 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2350811854 0 0) 0x7fc7a0003520 con 0x7fc7bc143930 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 msgr2=0x7fc7bc165360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 0x7fc7bc165360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 >> v1:172.21.15.132:6789/0 conn(0x7fc7bc143460 legacy=0x7fc7bc15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc0ccc30 con 0x7fc7bc143930 2024-03-23T07:56:57.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fc7a8001670 con 0x7fc7bc143930 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 >> v1:172.21.15.105:6789/0 conn(0x7fc7bc143930 legacy=0x7fc7bc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 0x7fc7a00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0003ee0 0x7fc7a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004d00 0x7fc7a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0cc950 con 0x7fc7a0004d00 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a000b9e0 con 0x7fc7a0003ee0 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.225+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a000bba0 con 0x7fc7bc0cde20 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7bb400640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004d00 0x7fc7a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7ba000640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0003ee0 0x7fc7a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7baa00640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 0x7fc7a00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7bb400640 1 -- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 msgr2=0x7fc7a00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7bb400640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0cde20 0x7fc7a00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7bb400640 1 -- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0003ee0 msgr2=0x7fc7a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:57.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7bb400640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0003ee0 0x7fc7a00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7bb400640 1 -- 172.21.15.105:0/2088895281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0003520 con 0x7fc7a0004d00 2024-03-23T07:56:57.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.226+0000 7fc7ba000640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0003ee0 0x7fc7a00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.227+0000 7fc7bb400640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004d00 0x7fc7a0005180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009fa0 tx=0x7fc7a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82d442ee6fd934a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b402ccb0 con 0x7fc7a0004d00 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b400f040 con 0x7fc7a0004d00 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7b9600640 1 -- 172.21.15.105:0/2088895281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4034660 con 0x7fc7a0004d00 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7bbe00640 1 -- 172.21.15.105:0/2088895281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004d00 msgr2=0x7fc7a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7bbe00640 1 --2- 172.21.15.105:0/2088895281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004d00 0x7fc7a0005180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009fa0 tx=0x7fc7a00057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7bb400640 1 -- 172.21.15.105:0/2088895281 reap_dead start 2024-03-23T07:56:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7bbe00640 1 -- 172.21.15.105:0/2088895281 shutdown_connections 2024-03-23T07:56:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7bbe00640 1 -- 172.21.15.105:0/2088895281 >> 172.21.15.105:0/2088895281 conn(0x7fc7bc065df0 msgr2=0x7fc7bc069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7bbe00640 1 -- 172.21.15.105:0/2088895281 shutdown_connections 2024-03-23T07:56:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.228+0000 7fc7bbe00640 1 -- 172.21.15.105:0/2088895281 wait complete. 2024-03-23T07:56:57.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.230+0000 7fc7bbe00640 1 Processor -- start 2024-03-23T07:56:57.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.230+0000 7fc7bbe00640 1 -- start start 2024-03-23T07:56:57.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0004d00 0x7fc7bc0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0b1ae0 0x7fc7bc0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc137ed0 0x7fc7bc138350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bbe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0b24a0 con 0x7fc7bc137ed0 2024-03-23T07:56:57.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc06cdc0 con 0x7fc7a0004d00 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc13a7b0 con 0x7fc7bc0b1ae0 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bb400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc137ed0 0x7fc7bc138350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bb400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc137ed0 0x7fc7bc138350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47436/0 (socket says 172.21.15.105:47436) 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.231+0000 7fc7bb400640 1 -- 172.21.15.105:0/591924350 learned_addr learned my addr 172.21.15.105:0/591924350 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7ba000640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0b1ae0 0x7fc7bc0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7baa00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0004d00 0x7fc7bc0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7bb400640 1 -- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0b1ae0 msgr2=0x7fc7bc0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7bb400640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0b1ae0 0x7fc7bc0b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7bb400640 1 -- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0004d00 msgr2=0x7fc7bc0cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7bb400640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0004d00 0x7fc7bc0cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7bb400640 1 -- 172.21.15.105:0/591924350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc13aa90 con 0x7fc7bc137ed0 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7bb400640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc137ed0 0x7fc7bc138350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4037040 tx=0x7fc7b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7b2a00640 1 -- 172.21.15.105:0/591924350 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4034cd0 con 0x7fc7bc137ed0 2024-03-23T07:56:57.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7b2a00640 1 -- 172.21.15.105:0/591924350 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b4034e50 con 0x7fc7bc137ed0 2024-03-23T07:56:57.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.232+0000 7fc7b2a00640 1 -- 172.21.15.105:0/591924350 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b403e650 con 0x7fc7bc137ed0 2024-03-23T07:56:57.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.233+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc138890 con 0x7fc7bc137ed0 2024-03-23T07:56:57.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.233+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc138dc0 con 0x7fc7bc137ed0 2024-03-23T07:56:57.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.234+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc04e030 con 0x7fc7bc137ed0 2024-03-23T07:56:57.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.234+0000 7fc7b2a00640 1 -- 172.21.15.105:0/591924350 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc7b403b050 con 0x7fc7bc137ed0 2024-03-23T07:56:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.236+0000 7fc7b2a00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc79c06ac00 0x7fc79c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.236+0000 7fc7b2a00640 1 -- 172.21.15.105:0/591924350 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc7b40ab830 con 0x7fc7bc137ed0 2024-03-23T07:56:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.236+0000 7fc7baa00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc79c06ac00 0x7fc79c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.237+0000 7fc7baa00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc79c06ac00 0x7fc79c06d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac003700 tx=0x7fc7ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:57.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.240+0000 7fc7b2a00640 1 -- 172.21.15.105:0/591924350 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc7b4076710 con 0x7fc7bc137ed0 2024-03-23T07:56:57.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.433+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7bc0b05d0 con 0x7fc7bc137ed0 2024-03-23T07:56:57.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.433+0000 7fc7b2a00640 1 -- 172.21.15.105:0/591924350 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7b4079d30 con 0x7fc7bc137ed0 2024-03-23T07:56:57.434 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc79c06ac00 msgr2=0x7fc79c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc79c06ac00 0x7fc79c06d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac003700 tx=0x7fc7ac009040 comp rx=0 tx=0).stop 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc137ed0 msgr2=0x7fc7bc138350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc137ed0 0x7fc7bc138350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4037040 tx=0x7fc7b4009750 comp rx=0 tx=0).stop 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 shutdown_connections 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7bc137ed0 0x7fc7bc138350 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc79c06ac00 0x7fc79c06d0c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7bc0b1ae0 0x7fc7bc0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 --2- 172.21.15.105:0/591924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a0004d00 0x7fc7bc0cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 >> 172.21.15.105:0/591924350 conn(0x7fc7bc065df0 msgr2=0x7fc7bc139680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 shutdown_connections 2024-03-23T07:56:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.436+0000 7fc7bbe00640 1 -- 172.21.15.105:0/591924350 wait complete. 2024-03-23T07:56:57.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:56:57.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.665+0000 7f0262200640 1 Processor -- start 2024-03-23T07:56:57.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.665+0000 7f0262200640 1 -- start start 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.666+0000 7f0262200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f025c0b1c50 0x7f025c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.666+0000 7f0262200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f025c0b2030 0x7f025c069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.666+0000 7f0262200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c0b33b0 con 0x7f025c0ca840 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.666+0000 7f0262200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c0b2f90 con 0x7f025c0b2030 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.666+0000 7f0262200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c159410 con 0x7f025c0b1c50 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025be00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f025c0b2030 0x7f025c069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025be00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f025c0b2030 0x7f025c069c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47452/0 (socket says 172.21.15.105:47452) 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025be00640 1 -- 172.21.15.105:0/402309176 learned_addr learned my addr 172.21.15.105:0/402309176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f0261800640 1 --1- 172.21.15.105:0/402309176 >> v1:172.21.15.105:6789/0 conn(0x7f025c0ca840 0x7f025c0cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33952/0 (socket says 172.21.15.105:33952) 2024-03-23T07:56:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f0260e00640 1 --2- 172.21.15.105:0/402309176 >> v2:172.21.15.132:3300/0 conn(0x7f025c0b1c50 0x7f025c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025be00640 1 -- 172.21.15.105:0/402309176 >> v2:172.21.15.132:3300/0 conn(0x7f025c0b1c50 msgr2=0x7f025c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025be00640 1 --2- 172.21.15.105:0/402309176 >> v2:172.21.15.132:3300/0 conn(0x7f025c0b1c50 0x7f025c0ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025be00640 1 -- 172.21.15.105:0/402309176 >> v1:172.21.15.105:6789/0 conn(0x7f025c0ca840 legacy=0x7f025c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025be00640 1 -- 172.21.15.105:0/402309176 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0b2bd0 con 0x7f025c0b2030 2024-03-23T07:56:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.667+0000 7f025b400640 1 -- 172.21.15.105:0/402309176 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531297335 0 0) 0x7f025c0b33b0 con 0x7f025c0ca840 2024-03-23T07:56:57.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.668+0000 7f025be00640 1 --2- 172.21.15.105:0/402309176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b2030 0x7f025c069c90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f025c0cace0 tx=0x7f024802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddacee1d91e3613 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:57.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.668+0000 7f025b400640 1 -- 172.21.15.105:0/402309176 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024802dba0 con 0x7f025c0b2030 2024-03-23T07:56:57.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f025b400640 1 -- 172.21.15.105:0/402309176 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024802dd00 con 0x7f025c0b2030 2024-03-23T07:56:57.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f025b400640 1 -- 172.21.15.105:0/402309176 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248035660 con 0x7f025c0b2030 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 -- 172.21.15.105:0/402309176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b2030 msgr2=0x7f025c069c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 --2- 172.21.15.105:0/402309176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b2030 0x7f025c069c90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f025c0cace0 tx=0x7f024802cdf0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 -- 172.21.15.105:0/402309176 shutdown_connections 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 --2- 172.21.15.105:0/402309176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b2030 0x7f025c069c90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 --2- 172.21.15.105:0/402309176 >> v2:172.21.15.132:3300/0 conn(0x7f025c0b1c50 0x7f025c0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 -- 172.21.15.105:0/402309176 >> 172.21.15.105:0/402309176 conn(0x7f025c0b01b0 msgr2=0x7f025c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 -- 172.21.15.105:0/402309176 shutdown_connections 2024-03-23T07:56:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.669+0000 7f0262200640 1 -- 172.21.15.105:0/402309176 wait complete. 2024-03-23T07:56:57.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.671+0000 7f0262200640 1 Processor -- start 2024-03-23T07:56:57.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.671+0000 7f0262200640 1 -- start start 2024-03-23T07:56:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0262200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b1c50 0x7f025c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0262200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f025c0b2030 0x7f025c0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0262200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f025c0ca840 0x7f025c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0262200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c1551c0 con 0x7f025c0b1c50 2024-03-23T07:56:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0262200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0983c0 con 0x7f025c0ca840 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0262200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0986a0 con 0x7f025c0b2030 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0260e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b1c50 0x7f025c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0260e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b1c50 0x7f025c154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47460/0 (socket says 172.21.15.105:47460) 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.672+0000 7f0260e00640 1 -- 172.21.15.105:0/2720851857 learned_addr learned my addr 172.21.15.105:0/2720851857 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0261800640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f025c0ca840 0x7f025c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0260e00640 1 -- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f025c0b2030 msgr2=0x7f025c0ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0260e00640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f025c0b2030 0x7f025c0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0260e00640 1 -- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f025c0ca840 msgr2=0x7f025c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0260e00640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f025c0ca840 0x7f025c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0260e00640 1 -- 172.21.15.105:0/2720851857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024800a670 con 0x7f025c0b1c50 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0261800640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f025c0ca840 0x7f025c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0260e00640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b1c50 0x7f025c154c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f025000e9b0 tx=0x7f025000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0258c00640 1 -- 172.21.15.105:0/2720851857 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250009670 con 0x7f025c0b1c50 2024-03-23T07:56:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0258c00640 1 -- 172.21.15.105:0/2720851857 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025000c9a0 con 0x7f025c0b1c50 2024-03-23T07:56:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c098980 con 0x7f025c0b1c50 2024-03-23T07:56:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0258c00640 1 -- 172.21.15.105:0/2720851857 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000cd50 con 0x7f025c0b1c50 2024-03-23T07:56:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.673+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c098f40 con 0x7f025c0b1c50 2024-03-23T07:56:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.674+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c04e030 con 0x7f025c0b1c50 2024-03-23T07:56:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.675+0000 7f0258c00640 1 -- 172.21.15.105:0/2720851857 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0250013470 con 0x7f025c0b1c50 2024-03-23T07:56:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.676+0000 7f0258c00640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f023c06ac30 0x7f023c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.676+0000 7f0258c00640 1 -- 172.21.15.105:0/2720851857 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0250089890 con 0x7f025c0b1c50 2024-03-23T07:56:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.677+0000 7f025be00640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f023c06ac30 0x7f023c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.677+0000 7f025be00640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f023c06ac30 0x7f023c06d0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0248037000 tx=0x7f024802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.681+0000 7f0258c00640 1 -- 172.21.15.105:0/2720851857 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0250054780 con 0x7f025c0b1c50 2024-03-23T07:56:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.878+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f025c158ea0 con 0x7f025c0b1c50 2024-03-23T07:56:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.879+0000 7f0258c00640 1 -- 172.21.15.105:0/2720851857 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0250057da0 con 0x7f025c0b1c50 2024-03-23T07:56:57.879 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:56:57.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f023c06ac30 msgr2=0x7f023c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f023c06ac30 0x7f023c06d0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0248037000 tx=0x7f024802f000 comp rx=0 tx=0).stop 2024-03-23T07:56:57.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b1c50 msgr2=0x7f025c154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:57.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b1c50 0x7f025c154c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f025000e9b0 tx=0x7f025000ee70 comp rx=0 tx=0).stop 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 shutdown_connections 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f025c0ca840 0x7f025c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f023c06ac30 0x7f023c06d0f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f025c0b2030 0x7f025c0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 --2- 172.21.15.105:0/2720851857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f025c0b1c50 0x7f025c154c80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.881+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 >> 172.21.15.105:0/2720851857 conn(0x7f025c0b01b0 msgr2=0x7f025c0c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.882+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 shutdown_connections 2024-03-23T07:56:57.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:57.882+0000 7f0262200640 1 -- 172.21.15.105:0/2720851857 wait complete. 2024-03-23T07:56:57.901 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:56:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.105+0000 7f9f07200640 1 Processor -- start 2024-03-23T07:56:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.106+0000 7f9f07200640 1 -- start start 2024-03-23T07:56:58.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.107+0000 7f9f07200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f000adbc0 0x7f9f000b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.107+0000 7f9f07200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9f000cbab0 0x7f9f000b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.107+0000 7f9f07200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9f0006d390 0x7f9f0006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.107+0000 7f9f07200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0006dd60 con 0x7f9f000cbab0 2024-03-23T07:56:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.107+0000 7f9f07200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f000b16a0 con 0x7f9f0006d390 2024-03-23T07:56:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.107+0000 7f9f07200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b4700 con 0x7f9f000adbc0 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9f000cbab0 0x7f9f000b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9f000cbab0 0x7f9f000b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47474/0 (socket says 172.21.15.105:47474) 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 -- 172.21.15.105:0/4069560114 learned_addr learned my addr 172.21.15.105:0/4069560114 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9f05200640 1 --2- 172.21.15.105:0/4069560114 >> v2:172.21.15.132:3300/0 conn(0x7f9f0006d390 0x7f9f0006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9effe00640 1 --2- 172.21.15.105:0/4069560114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f000adbc0 0x7f9f000b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 -- 172.21.15.105:0/4069560114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f000adbc0 msgr2=0x7f9f000b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 --2- 172.21.15.105:0/4069560114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f000adbc0 0x7f9f000b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 -- 172.21.15.105:0/4069560114 >> v2:172.21.15.132:3300/0 conn(0x7f9f0006d390 msgr2=0x7f9f0006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 --2- 172.21.15.105:0/4069560114 >> v2:172.21.15.132:3300/0 conn(0x7f9f0006d390 0x7f9f0006d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.108+0000 7f9ef7400640 1 -- 172.21.15.105:0/4069560114 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000cbfb0 con 0x7f9f000cbab0 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.109+0000 7f9ef7400640 1 --2- 172.21.15.105:0/4069560114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000cbab0 0x7f9f000b20f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f000cc650 tx=0x7f9ee802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7b2fb8510ab5d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.109+0000 7f9eff400640 1 -- 172.21.15.105:0/4069560114 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee80317d0 con 0x7f9f000cbab0 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.109+0000 7f9eff400640 1 -- 172.21.15.105:0/4069560114 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8031930 con 0x7f9f000cbab0 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.109+0000 7f9eff400640 1 -- 172.21.15.105:0/4069560114 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8031cc0 con 0x7f9f000cbab0 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 -- 172.21.15.105:0/4069560114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000cbab0 msgr2=0x7f9f000b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 --2- 172.21.15.105:0/4069560114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000cbab0 0x7f9f000b20f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f000cc650 tx=0x7f9ee802b870 comp rx=0 tx=0).stop 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 -- 172.21.15.105:0/4069560114 shutdown_connections 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 --2- 172.21.15.105:0/4069560114 >> v2:172.21.15.132:3300/0 conn(0x7f9f0006d390 0x7f9f0006d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 --2- 172.21.15.105:0/4069560114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000cbab0 0x7f9f000b20f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 --2- 172.21.15.105:0/4069560114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f000adbc0 0x7f9f000b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 -- 172.21.15.105:0/4069560114 >> 172.21.15.105:0/4069560114 conn(0x7f9f0006a3d0 msgr2=0x7f9f000b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 -- 172.21.15.105:0/4069560114 shutdown_connections 2024-03-23T07:56:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.110+0000 7f9f07200640 1 -- 172.21.15.105:0/4069560114 wait complete. 2024-03-23T07:56:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.112+0000 7f9f07200640 1 Processor -- start 2024-03-23T07:56:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.112+0000 7f9f07200640 1 -- start start 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9f07200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f0006d390 0x7f9f001efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9f07200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000adbc0 0x7f9f000ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9f07200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f000cbab0 0x7f9f000aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9f07200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000a8d30 con 0x7f9f000adbc0 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9f07200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000a9010 con 0x7f9f000cbab0 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9f07200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f001328f0 con 0x7f9f0006d390 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9ef7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000adbc0 0x7f9f000ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9ef7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000adbc0 0x7f9f000ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47482/0 (socket says 172.21.15.105:47482) 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9ef7400640 1 -- 172.21.15.105:0/233227174 learned_addr learned my addr 172.21.15.105:0/233227174 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.113+0000 7f9effe00640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f0006d390 0x7f9f001efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9f05200640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f000cbab0 0x7f9f000aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9ef7400640 1 -- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f0006d390 msgr2=0x7f9f001efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9ef7400640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f0006d390 0x7f9f001efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9ef7400640 1 -- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f000cbab0 msgr2=0x7f9f000aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9ef7400640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f000cbab0 0x7f9f000aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9ef7400640 1 -- 172.21.15.105:0/233227174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee80094d0 con 0x7f9f000adbc0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9effe00640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f0006d390 0x7f9f001efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9ef7400640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000adbc0 0x7f9f000ac540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80090b0 tx=0x7f9ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9efcc00640 1 -- 172.21.15.105:0/233227174 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8032870 con 0x7f9f000adbc0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9efcc00640 1 -- 172.21.15.105:0/233227174 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee80329d0 con 0x7f9f000adbc0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9efcc00640 1 -- 172.21.15.105:0/233227174 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8032d80 con 0x7f9f000adbc0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.114+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f00132bd0 con 0x7f9f000adbc0 2024-03-23T07:56:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.115+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f001eff40 con 0x7f9f000adbc0 2024-03-23T07:56:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.115+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f001330f0 con 0x7f9f000adbc0 2024-03-23T07:56:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.116+0000 7f9efcc00640 1 -- 172.21.15.105:0/233227174 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ee8030070 con 0x7f9f000adbc0 2024-03-23T07:56:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.117+0000 7f9efcc00640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9edc06ac00 0x7f9edc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.117+0000 7f9efcc00640 1 -- 172.21.15.105:0/233227174 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ee80ab170 con 0x7f9f000adbc0 2024-03-23T07:56:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.117+0000 7f9effe00640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9edc06ac00 0x7f9edc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.118+0000 7f9effe00640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9edc06ac00 0x7f9edc06d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.122+0000 7f9efcc00640 1 -- 172.21.15.105:0/233227174 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f001330f0 con 0x7f9f000adbc0 2024-03-23T07:56:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.355+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f001330f0 con 0x7f9f000adbc0 2024-03-23T07:56:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.355+0000 7f9efcc00640 1 -- 172.21.15.105:0/233227174 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7f9f001330f0 con 0x7f9f000adbc0 2024-03-23T07:56:58.356 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:58.356 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:56:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.358+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9edc06ac00 msgr2=0x7f9edc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.358+0000 7f9f07200640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9edc06ac00 0x7f9edc06d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0008040 comp rx=0 tx=0).stop 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000adbc0 msgr2=0x7f9f000ac540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000adbc0 0x7f9f000ac540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80090b0 tx=0x7f9ee80046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 shutdown_connections 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f000cbab0 0x7f9f000aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f000adbc0 0x7f9f000ac540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9edc06ac00 0x7f9edc06d0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 --2- 172.21.15.105:0/233227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f0006d390 0x7f9f001efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 >> 172.21.15.105:0/233227174 conn(0x7f9f0006a3d0 msgr2=0x7f9f000b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 shutdown_connections 2024-03-23T07:56:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.359+0000 7f9f07200640 1 -- 172.21.15.105:0/233227174 wait complete. 2024-03-23T07:56:58.376 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:56:58.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.570+0000 7f8766600640 1 Processor -- start 2024-03-23T07:56:58.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.571+0000 7f8766600640 1 -- start start 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f8766600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f87600b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f8766600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f876006d390 0x7f876006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f8766600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87600b16a0 con 0x7f87600adbc0 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f8766600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876006dd60 con 0x7f876006d390 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f8766600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600cbfb0 con 0x7f87600cbab0 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f875f400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f87600adbc0 0x7f87600b41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33962/0 (socket says 172.21.15.105:33962) 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f875f400640 1 -- 172.21.15.105:0/3629378740 learned_addr learned my addr 172.21.15.105:0/3629378740 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.572+0000 7f875fe00640 1 --2- 172.21.15.105:0/3629378740 >> v2:172.21.15.105:3300/0 conn(0x7f876006d390 0x7f876006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f875ea00640 1 -- 172.21.15.105:0/3629378740 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483436432 0 0) 0x7f87600b16a0 con 0x7f87600adbc0 2024-03-23T07:56:58.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f875ea00640 1 -- 172.21.15.105:0/3629378740 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8744003520 con 0x7f87600adbc0 2024-03-23T07:56:58.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f8756a00640 1 --2- 172.21.15.105:0/3629378740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f87600b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f875fe00640 1 -- 172.21.15.105:0/3629378740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 msgr2=0x7f87600b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:58.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f875fe00640 1 --2- 172.21.15.105:0/3629378740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f87600b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f875fe00640 1 -- 172.21.15.105:0/3629378740 >> v1:172.21.15.105:6789/0 conn(0x7f87600adbc0 legacy=0x7f87600b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f875fe00640 1 -- 172.21.15.105:0/3629378740 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600cc290 con 0x7f876006d390 2024-03-23T07:56:58.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.573+0000 7f8756a00640 1 --2- 172.21.15.105:0/3629378740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f87600b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:58.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.574+0000 7f875fe00640 1 --2- 172.21.15.105:0/3629378740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876006d820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8758009d90 tx=0x7f875802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5e75555d5a7515 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:58.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.575+0000 7f875ea00640 1 -- 172.21.15.105:0/3629378740 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875802cbd0 con 0x7f876006d390 2024-03-23T07:56:58.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.575+0000 7f875ea00640 1 -- 172.21.15.105:0/3629378740 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875802cd30 con 0x7f876006d390 2024-03-23T07:56:58.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.575+0000 7f875ea00640 1 -- 172.21.15.105:0/3629378740 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875802cea0 con 0x7f876006d390 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.575+0000 7f8766600640 1 -- 172.21.15.105:0/3629378740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 msgr2=0x7f876006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.575+0000 7f8766600640 1 --2- 172.21.15.105:0/3629378740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876006d820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8758009d90 tx=0x7f875802be20 comp rx=0 tx=0).stop 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.576+0000 7f8766600640 1 -- 172.21.15.105:0/3629378740 shutdown_connections 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.576+0000 7f8766600640 1 --2- 172.21.15.105:0/3629378740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876006d820 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.576+0000 7f8766600640 1 --2- 172.21.15.105:0/3629378740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f87600b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.576+0000 7f8766600640 1 -- 172.21.15.105:0/3629378740 >> 172.21.15.105:0/3629378740 conn(0x7f876006a3d0 msgr2=0x7f87600b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.576+0000 7f8766600640 1 -- 172.21.15.105:0/3629378740 shutdown_connections 2024-03-23T07:56:58.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.576+0000 7f8766600640 1 -- 172.21.15.105:0/3629378740 wait complete. 2024-03-23T07:56:58.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.577+0000 7f8766600640 1 Processor -- start 2024-03-23T07:56:58.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.577+0000 7f8766600640 1 -- start start 2024-03-23T07:56:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.578+0000 7f8766600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.578+0000 7f8766600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87600adbc0 0x7f8760139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.578+0000 7f8766600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f8760158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.578+0000 7f8766600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876013cc00 con 0x7f876006d390 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.578+0000 7f8766600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876013cee0 con 0x7f87600adbc0 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.578+0000 7f8766600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876013d1c0 con 0x7f87600cbab0 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876013c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47502/0 (socket says 172.21.15.105:47502) 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 -- 172.21.15.105:0/1967667273 learned_addr learned my addr 172.21.15.105:0/1967667273 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875fe00640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f8760158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f8756a00640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87600adbc0 0x7f8760139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 -- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 msgr2=0x7f8760158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f8760158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 -- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87600adbc0 msgr2=0x7f8760139cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87600adbc0 0x7f8760139cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.579+0000 7f875f400640 1 -- 172.21.15.105:0/1967667273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009a70 con 0x7f876006d390 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.580+0000 7f875f400640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876013c620 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f875000c430 tx=0x7f875000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.580+0000 7f8757e00640 1 -- 172.21.15.105:0/1967667273 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750005ce0 con 0x7f876006d390 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.580+0000 7f8757e00640 1 -- 172.21.15.105:0/1967667273 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750005e40 con 0x7f876006d390 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.580+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876013d4a0 con 0x7f876006d390 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.580+0000 7f8757e00640 1 -- 172.21.15.105:0/1967667273 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87500072f0 con 0x7f876006d390 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.580+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876013a1f0 con 0x7f876006d390 2024-03-23T07:56:58.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.581+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876013d9c0 con 0x7f876006d390 2024-03-23T07:56:58.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.583+0000 7f8757e00640 1 -- 172.21.15.105:0/1967667273 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8750007450 con 0x7f876006d390 2024-03-23T07:56:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.584+0000 7f8757e00640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f873c06ac00 0x7f873c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.585+0000 7f8757e00640 1 -- 172.21.15.105:0/1967667273 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8750088de0 con 0x7f876006d390 2024-03-23T07:56:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.585+0000 7f8756a00640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f873c06ac00 0x7f873c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.586+0000 7f8756a00640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f873c06ac00 0x7f873c06d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87600b3370 tx=0x7f8748008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:58.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.590+0000 7f8757e00640 1 -- 172.21.15.105:0/1967667273 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f876013d9c0 con 0x7f876006d390 2024-03-23T07:56:58.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.787+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f876013d9c0 con 0x7f876006d390 2024-03-23T07:56:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.790+0000 7f8757e00640 1 -- 172.21.15.105:0/1967667273 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17930 (secure 0 0 0) 0x7f876013d9c0 con 0x7f876006d390 2024-03-23T07:56:58.791 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:58.791 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:22.456764+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:56:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f873c06ac00 msgr2=0x7f873c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f873c06ac00 0x7f873c06d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87600b3370 tx=0x7f8748008040 comp rx=0 tx=0).stop 2024-03-23T07:56:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 msgr2=0x7f876013c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876013c620 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f875000c430 tx=0x7f875000c8f0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 shutdown_connections 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f873c06ac00 0x7f873c06d0c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87600cbab0 0x7f8760158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87600adbc0 0x7f8760139cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 --2- 172.21.15.105:0/1967667273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f876006d390 0x7f876013c620 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.793+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 >> 172.21.15.105:0/1967667273 conn(0x7f876006a3d0 msgr2=0x7f876015db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.794+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 shutdown_connections 2024-03-23T07:56:58.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:58.794+0000 7f8766600640 1 -- 172.21.15.105:0/1967667273 wait complete. 2024-03-23T07:56:58.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:56:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.024+0000 7f29afc00640 1 Processor -- start 2024-03-23T07:56:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.024+0000 7f29afc00640 1 -- start start 2024-03-23T07:56:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.025+0000 7f29afc00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29a815b560 0x7f29a815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29afc00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a815f320 con 0x7f29a815d410 2024-03-23T07:56:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29afc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a806c6f0 con 0x7f29a80b39b0 2024-03-23T07:56:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29afc00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a806c9d0 con 0x7f29a815b560 2024-03-23T07:56:59.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29ad200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29a815b560 0x7f29a815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29ad200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29a815b560 0x7f29a815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47508/0 (socket says 172.21.15.105:47508) 2024-03-23T07:56:59.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29ad200640 1 -- 172.21.15.105:0/3128926560 learned_addr learned my addr 172.21.15.105:0/3128926560 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:59.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29adc00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f29a815d410 0x7f29a815ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33970/0 (socket says 172.21.15.105:33970) 2024-03-23T07:56:59.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29ad200640 1 -- 172.21.15.105:0/3128926560 >> v1:172.21.15.132:6789/0 conn(0x7f29a80b39b0 legacy=0x7f29a815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29ad200640 1 -- 172.21.15.105:0/3128926560 >> v1:172.21.15.105:6789/0 conn(0x7f29a815d410 legacy=0x7f29a815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.026+0000 7f29ad200640 1 -- 172.21.15.105:0/3128926560 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a806ccb0 con 0x7f29a815b560 2024-03-23T07:56:59.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.027+0000 7f29ad200640 1 --2- 172.21.15.105:0/3128926560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a815b560 0x7f29a815b940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2998009870 tx=0x7f299802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ff93d26286c3d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.028+0000 7f29a7400640 1 -- 172.21.15.105:0/3128926560 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998031860 con 0x7f29a815b560 2024-03-23T07:56:59.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.028+0000 7f29a7400640 1 -- 172.21.15.105:0/3128926560 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980319c0 con 0x7f29a815b560 2024-03-23T07:56:59.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.028+0000 7f29a7400640 1 -- 172.21.15.105:0/3128926560 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998031d50 con 0x7f29a815b560 2024-03-23T07:56:59.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.028+0000 7f29afc00640 1 -- 172.21.15.105:0/3128926560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a815b560 msgr2=0x7f29a815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.028+0000 7f29afc00640 1 --2- 172.21.15.105:0/3128926560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a815b560 0x7f29a815b940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2998009870 tx=0x7f299802b900 comp rx=0 tx=0).stop 2024-03-23T07:56:59.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.029+0000 7f29afc00640 1 -- 172.21.15.105:0/3128926560 shutdown_connections 2024-03-23T07:56:59.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.029+0000 7f29afc00640 1 --2- 172.21.15.105:0/3128926560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a815b560 0x7f29a815b940 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.029+0000 7f29afc00640 1 -- 172.21.15.105:0/3128926560 >> 172.21.15.105:0/3128926560 conn(0x7f29a806a3d0 msgr2=0x7f29a80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.029+0000 7f29afc00640 1 -- 172.21.15.105:0/3128926560 shutdown_connections 2024-03-23T07:56:59.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.029+0000 7f29afc00640 1 -- 172.21.15.105:0/3128926560 wait complete. 2024-03-23T07:56:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.030+0000 7f29afc00640 1 Processor -- start 2024-03-23T07:56:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.030+0000 7f29afc00640 1 -- start start 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.031+0000 7f29afc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a80b39b0 0x7f29a813dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.031+0000 7f29afc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a815b560 0x7f29a813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.031+0000 7f29afc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a815d410 0x7f29a8140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.031+0000 7f29afc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8157dd0 con 0x7f29a80b39b0 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.031+0000 7f29afc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80aebb0 con 0x7f29a815b560 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.031+0000 7f29afc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8158350 con 0x7f29a815d410 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a80b39b0 0x7f29a813dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a80b39b0 0x7f29a813dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47524/0 (socket says 172.21.15.105:47524) 2024-03-23T07:56:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 -- 172.21.15.105:0/486579002 learned_addr learned my addr 172.21.15.105:0/486579002 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29a7e00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a815b560 0x7f29a813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 -- 172.21.15.105:0/486579002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a815d410 msgr2=0x7f29a8140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a815d410 0x7f29a8140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 -- 172.21.15.105:0/486579002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a815b560 msgr2=0x7f29a813fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a815b560 0x7f29a813fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 -- 172.21.15.105:0/486579002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980094d0 con 0x7f29a80b39b0 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.032+0000 7f29ad200640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a80b39b0 0x7f29a813dcc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2998009870 tx=0x7f29980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.033+0000 7f29a4c00640 1 -- 172.21.15.105:0/486579002 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998031810 con 0x7f29a80b39b0 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.033+0000 7f29a4c00640 1 -- 172.21.15.105:0/486579002 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980443f0 con 0x7f29a80b39b0 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.033+0000 7f29a4c00640 1 -- 172.21.15.105:0/486579002 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980447a0 con 0x7f29a80b39b0 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.033+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a8158070 con 0x7f29a80b39b0 2024-03-23T07:56:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.033+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a80af130 con 0x7f29a80b39b0 2024-03-23T07:56:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.034+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a806ac20 con 0x7f29a80b39b0 2024-03-23T07:56:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.035+0000 7f29a4c00640 1 -- 172.21.15.105:0/486579002 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f299804d450 con 0x7f29a80b39b0 2024-03-23T07:56:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.036+0000 7f29a4c00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f298806ac00 0x7f298806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.036+0000 7f29a4c00640 1 -- 172.21.15.105:0/486579002 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f29980abe20 con 0x7f29a80b39b0 2024-03-23T07:56:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.037+0000 7f29a7e00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f298806ac00 0x7f298806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.037+0000 7f29a7e00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f298806ac00 0x7f298806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2994003d90 tx=0x7f2994009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.041+0000 7f29a4c00640 1 -- 172.21.15.105:0/486579002 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2998076cb0 con 0x7f29a80b39b0 2024-03-23T07:56:59.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.274+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29a80af320 con 0x7f29a80b39b0 2024-03-23T07:56:59.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.275+0000 7f29a4c00640 1 -- 172.21.15.105:0/486579002 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7f2998035020 con 0x7f29a80b39b0 2024-03-23T07:56:59.276 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:56:59.276 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:56:59.276 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:56:59.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.277+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f298806ac00 msgr2=0x7f298806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.277+0000 7f29afc00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f298806ac00 0x7f298806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2994003d90 tx=0x7f2994009040 comp rx=0 tx=0).stop 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.277+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a80b39b0 msgr2=0x7f29a813dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.277+0000 7f29afc00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a80b39b0 0x7f29a813dcc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2998009870 tx=0x7f29980046c0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 shutdown_connections 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f298806ac00 0x7f298806d0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a815d410 0x7f29a8140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a815b560 0x7f29a813fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 --2- 172.21.15.105:0/486579002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a80b39b0 0x7f29a813dcc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 >> 172.21.15.105:0/486579002 conn(0x7f29a806a3d0 msgr2=0x7f29a8158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 shutdown_connections 2024-03-23T07:56:59.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.278+0000 7f29afc00640 1 -- 172.21.15.105:0/486579002 wait complete. 2024-03-23T07:56:59.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T07:56:17.694756+0000', 'modified': '2024-03-23T07:56:45.123427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6834/713904573', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 713904573}, {'type': 'v1', 'addr': '172.21.15.132:6834', 'nonce': 713904573}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T07:56:59.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T07:56:59.701 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T07:56:59.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.913+0000 7f9f65a00640 1 Processor -- start 2024-03-23T07:56:59.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.913+0000 7f9f65a00640 1 -- start start 2024-03-23T07:56:59.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.914+0000 7f9f65a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9f600ce930 0x7f9f60165340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.914+0000 7f9f65a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6006cdc0 con 0x7f9f601612d0 2024-03-23T07:56:59.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.914+0000 7f9f65a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f60161ab0 con 0x7f9f601627c0 2024-03-23T07:56:59.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.914+0000 7f9f65a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f60161d90 con 0x7f9f600ce930 2024-03-23T07:56:59.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.914+0000 7f9f57e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9f601612d0 0x7f9f600ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33980/0 (socket says 172.21.15.105:33980) 2024-03-23T07:56:59.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.914+0000 7f9f57e00640 1 -- 172.21.15.105:0/441109266 learned_addr learned my addr 172.21.15.105:0/441109266 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:56:59.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/441109266 >> v2:172.21.15.132:3300/0 conn(0x7f9f600ce930 0x7f9f60165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629855718 0 0) 0x7f9f6006cdc0 con 0x7f9f601612d0 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f44003520 con 0x7f9f601612d0 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3170612436 0 0) 0x7f9f44003520 con 0x7f9f601612d0 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 >> v2:172.21.15.132:3300/0 conn(0x7f9f600ce930 msgr2=0x7f9f60165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 --2- 172.21.15.105:0/441109266 >> v2:172.21.15.132:3300/0 conn(0x7f9f600ce930 0x7f9f60165340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 >> v1:172.21.15.132:6789/0 conn(0x7f9f601627c0 legacy=0x7f9f6006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f60162070 con 0x7f9f601612d0 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f9f48001670 con 0x7f9f601612d0 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 >> v1:172.21.15.105:6789/0 conn(0x7f9f601612d0 legacy=0x7f9f600ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f440039c0 0x7f9f44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440051d0 0x7f9f4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60161d90 con 0x7f9f440047c0 2024-03-23T07:56:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c100 con 0x7f9f440039c0 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.915+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c2c0 con 0x7f9f440051d0 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f57e00640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f440039c0 0x7f9f44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f5f400640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440051d0 0x7f9f4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f57e00640 1 -- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440051d0 msgr2=0x7f9f4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f57e00640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440051d0 0x7f9f4400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f57e00640 1 -- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f440039c0 msgr2=0x7f9f44003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f57e00640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f440039c0 0x7f9f44003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f57e00640 1 -- 172.21.15.105:0/441109266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44003520 con 0x7f9f440047c0 2024-03-23T07:56:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.916+0000 7f9f5f400640 1 -- 172.21.15.105:0/441109266 reap_dead start 2024-03-23T07:56:59.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.917+0000 7f9f57e00640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800c160 tx=0x7f9f440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c46c96765868d13 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.917+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48005ad0 con 0x7f9f440047c0 2024-03-23T07:56:59.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.917+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48005c30 con 0x7f9f440047c0 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.917+0000 7f9f5e000640 1 -- 172.21.15.105:0/441109266 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4802c7c0 con 0x7f9f440047c0 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.918+0000 7f9f65a00640 1 -- 172.21.15.105:0/441109266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f440047c0 msgr2=0x7f9f44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.918+0000 7f9f65a00640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800c160 tx=0x7f9f440057f0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.918+0000 7f9f65a00640 1 -- 172.21.15.105:0/441109266 shutdown_connections 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.918+0000 7f9f65a00640 1 --2- 172.21.15.105:0/441109266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.918+0000 7f9f65a00640 1 -- 172.21.15.105:0/441109266 >> 172.21.15.105:0/441109266 conn(0x7f9f6006a0b0 msgr2=0x7f9f6006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.918+0000 7f9f65a00640 1 -- 172.21.15.105:0/441109266 shutdown_connections 2024-03-23T07:56:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.918+0000 7f9f65a00640 1 -- 172.21.15.105:0/441109266 wait complete. 2024-03-23T07:56:59.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.920+0000 7f9f65a00640 1 Processor -- start 2024-03-23T07:56:59.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.920+0000 7f9f65a00640 1 -- start start 2024-03-23T07:56:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f65a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440047c0 0x7f9f6014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f65a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6014d2d0 0x7f9f601f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f65a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f601f6c30 0x7f9f601f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f65a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6014e580 con 0x7f9f6014d2d0 2024-03-23T07:56:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f65a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6014d750 con 0x7f9f601f6c30 2024-03-23T07:56:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f65a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601f75f0 con 0x7f9f440047c0 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6014d2d0 0x7f9f601f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6014d2d0 0x7f9f601f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47532/0 (socket says 172.21.15.105:47532) 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.921+0000 7f9f57e00640 1 -- 172.21.15.105:0/3324162897 learned_addr learned my addr 172.21.15.105:0/3324162897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f5f400640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f601f6c30 0x7f9f601f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440047c0 0x7f9f6014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f57e00640 1 -- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440047c0 msgr2=0x7f9f6014cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f57e00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440047c0 0x7f9f6014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f57e00640 1 -- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f601f6c30 msgr2=0x7f9f601f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f57e00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f601f6c30 0x7f9f601f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f57e00640 1 -- 172.21.15.105:0/3324162897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6006d840 con 0x7f9f6014d2d0 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f5f400640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f601f6c30 0x7f9f601f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:56:59.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f57e00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6014d2d0 0x7f9f601f66f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800c280 tx=0x7f9f4802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f56a00640 1 -- 172.21.15.105:0/3324162897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4803c040 con 0x7f9f6014d2d0 2024-03-23T07:56:59.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f56a00640 1 -- 172.21.15.105:0/3324162897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4802ccb0 con 0x7f9f6014d2d0 2024-03-23T07:56:59.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f56a00640 1 -- 172.21.15.105:0/3324162897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480072b0 con 0x7f9f6014d2d0 2024-03-23T07:56:59.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6014d970 con 0x7f9f6014d2d0 2024-03-23T07:56:59.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.922+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6014df30 con 0x7f9f6014d2d0 2024-03-23T07:56:59.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.924+0000 7f9f56a00640 1 -- 172.21.15.105:0/3324162897 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f48032070 con 0x7f9f6014d2d0 2024-03-23T07:56:59.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.925+0000 7f9f56a00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f3c06ac30 0x7f9f3c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.925+0000 7f9f56a00640 1 -- 172.21.15.105:0/3324162897 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f480aa970 con 0x7f9f6014d2d0 2024-03-23T07:56:59.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.926+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f3c06ac30 0x7f9f3c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.926+0000 7f9f687e7740 1 Processor -- start 2024-03-23T07:56:59.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.926+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f3c06ac30 0x7f9f3c06d0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f60074b10 tx=0x7f9f50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.926+0000 7f9f687e7740 1 -- start start 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.927+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd3bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.927+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 0x55a87bd6ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.927+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd3c1e0 0x55a87bd6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.927+0000 7f9f687e7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bd172c0 con 0x55a87bd68d00 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.927+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bcac8f0 con 0x55a87bd3c1e0 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.927+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a87bcaa080 con 0x55a87bd690e0 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd3bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd3bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47538/0 (socket says 172.21.15.105:47538) 2024-03-23T07:56:59.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 -- 172.21.15.105:0/686420596 learned_addr learned my addr 172.21.15.105:0/686420596 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 -- 172.21.15.105:0/686420596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 msgr2=0x55a87bd6ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/686420596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 0x55a87bd6ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 -- 172.21.15.105:0/686420596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd3c1e0 msgr2=0x55a87bd6bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/686420596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd3c1e0 0x55a87bd6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 -- 172.21.15.105:0/686420596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87bc70a10 con 0x55a87bd68d00 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.928+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/686420596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd3bca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f500108a0 tx=0x7f9f50013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1b5a127500305d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f4f400640 1 -- 172.21.15.105:0/686420596 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50015bf0 con 0x55a87bd68d00 2024-03-23T07:56:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f4f400640 1 -- 172.21.15.105:0/686420596 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5001a070 con 0x55a87bd68d00 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f4f400640 1 -- 172.21.15.105:0/686420596 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50021440 con 0x55a87bd68d00 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 -- 172.21.15.105:0/686420596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 msgr2=0x55a87bd3bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 --2- 172.21.15.105:0/686420596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd3bca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f500108a0 tx=0x7f9f50013000 comp rx=0 tx=0).stop 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 -- 172.21.15.105:0/686420596 shutdown_connections 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 --2- 172.21.15.105:0/686420596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd3c1e0 0x55a87bd6bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 --2- 172.21.15.105:0/686420596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 0x55a87bd6ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 --2- 172.21.15.105:0/686420596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd3bca0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 -- 172.21.15.105:0/686420596 >> 172.21.15.105:0/686420596 conn(0x55a87bd3df10 msgr2=0x55a87bd86a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 -- 172.21.15.105:0/686420596 shutdown_connections 2024-03-23T07:56:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.929+0000 7f9f687e7740 1 -- 172.21.15.105:0/686420596 wait complete. 2024-03-23T07:56:59.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.930+0000 7f9f687e7740 1 -- start start 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.930+0000 7f9f687e7740 1 Processor -- start 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.931+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bd6ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.931+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bd55a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.931+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 0x55a87bd54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.931+0000 7f9f687e7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bca5970 con 0x55a87bd690e0 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.931+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bc470a0 con 0x55a87bd68d00 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.931+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a87bc46d40 con 0x55a87bd525d0 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.931+0000 7f9f687e7740 10 client.0 _ll_register_callbacks cb 0x55a87bc9efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bd55a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bd55a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47552/0 (socket says 172.21.15.105:47552) 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 -- 172.21.15.105:0/3820411099 learned_addr learned my addr 172.21.15.105:0/3820411099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bd6ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 -- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 msgr2=0x55a87bd54510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 0x55a87bd54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 -- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 msgr2=0x55a87bd6ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bd6ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 -- 172.21.15.105:0/3820411099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5000e050 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f57e00640 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bd55a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f48075ad0 tx=0x7f9f4802cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53656cb781f4afaf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f43e00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f43e00640 1 -- 172.21.15.105:0/3820411099 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48091020 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.932+0000 7f9f43e00640 1 -- 172.21.15.105:0/3820411099 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f480819c0 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a87bccfb10 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f43e00640 1 -- 172.21.15.105:0/3820411099 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f48075ce0 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 10 client.4791 fetch_fsmap learned FSMap version 8 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a87bca3ff0 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f43e00640 1 -- 172.21.15.105:0/3820411099 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48085080 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f43e00640 1 -- 172.21.15.105:0/3820411099 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f9f4807d030 con 0x55a87bd690e0 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 10 client.4791 fetch_fsmap finished waiting for FSMap version 8 2024-03-23T07:56:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 10 client.4791 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T07:56:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 20 client.4791 populate_metadata read hostname 'smithi105' 2024-03-23T07:56:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bd67a10 0x55a87bd6aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 4 client.4791 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T07:56:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.933+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 --> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a87b9993c0 con 0x55a87bd67a10 2024-03-23T07:56:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.934+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bd67a10 0x55a87bd6aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.935+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bd67a10 0x55a87bd6aac0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f37536315abd000 server_cookie=fbe6172a8e61464e in_seq=0 out_seq=0 2024-03-23T07:56:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.935+0000 7f9f43e00640 10 client.4791 ms_handle_connect on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.936+0000 7f9f43e00640 1 -- 172.21.15.105:0/3820411099 <== mds.0 v2:172.21.15.132:6832/713904573 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a87b9993c0 con 0x55a87bd67a10 2024-03-23T07:56:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.936+0000 7f9f43e00640 10 client.4791 handle_command_reply: tid=0 2024-03-23T07:56:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.936+0000 7f9f687e7740 1 client.4791 shutdown 2024-03-23T07:56:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 msgr2=0x55a87bd55a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bd55a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f48075ad0 tx=0x7f9f4802cf60 comp rx=0 tx=0).stop 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 shutdown_connections 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 0x55a87bd54510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bd67a10 0x55a87bd6aac0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bd55a10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 --2- 172.21.15.105:0/3820411099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bd6ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f43e00640 0 client.4791 ms_handle_reset on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 >> 172.21.15.105:0/3820411099 conn(0x55a87bd3c1e0 msgr2=0x55a87bd89fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 shutdown_connections 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 1 -- 172.21.15.105:0/3820411099 wait complete. 2024-03-23T07:56:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.937+0000 7f9f687e7740 20 client.4791 trim_cache size 0 max 16384 2024-03-23T07:56:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.953+0000 7f9f687e7740 1 Processor -- start 2024-03-23T07:56:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.953+0000 7f9f687e7740 1 -- start start 2024-03-23T07:56:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd525d0 0x55a87bd54f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd67020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 0x55a87bd8a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f687e7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87ba0cf20 con 0x55a87bd68d00 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bc6c090 con 0x55a87bd525d0 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a87bd3c6f0 con 0x55a87bd690e0 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd67020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd67020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47566/0 (socket says 172.21.15.105:47566) 2024-03-23T07:56:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.954+0000 7f9f57e00640 1 -- 172.21.15.105:0/714110058 learned_addr learned my addr 172.21.15.105:0/714110058 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f57e00640 1 -- 172.21.15.105:0/714110058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 msgr2=0x55a87bd8a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f57e00640 1 --2- 172.21.15.105:0/714110058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 0x55a87bd8a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f57e00640 1 -- 172.21.15.105:0/714110058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd525d0 msgr2=0x55a87bd54f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f57e00640 1 --2- 172.21.15.105:0/714110058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd525d0 0x55a87bd54f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f57e00640 1 -- 172.21.15.105:0/714110058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87bcd97c0 con 0x55a87bd68d00 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f57e00640 1 --2- 172.21.15.105:0/714110058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd67020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f4802c610 tx=0x7f9f480aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6e8efb7f22c750 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f4fe00640 1 -- 172.21.15.105:0/714110058 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480819c0 con 0x55a87bd68d00 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f4fe00640 1 -- 172.21.15.105:0/714110058 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87bccfb10 con 0x55a87bd68d00 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.955+0000 7f9f4fe00640 1 -- 172.21.15.105:0/714110058 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4807d030 con 0x55a87bd68d00 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 -- 172.21.15.105:0/714110058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 msgr2=0x55a87bd67020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 --2- 172.21.15.105:0/714110058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd67020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f4802c610 tx=0x7f9f480aa000 comp rx=0 tx=0).stop 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 -- 172.21.15.105:0/714110058 shutdown_connections 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 --2- 172.21.15.105:0/714110058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd690e0 0x55a87bd8a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 --2- 172.21.15.105:0/714110058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd68d00 0x55a87bd67020 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 --2- 172.21.15.105:0/714110058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd525d0 0x55a87bd54f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 -- 172.21.15.105:0/714110058 >> 172.21.15.105:0/714110058 conn(0x55a87bd3c1e0 msgr2=0x55a87bd639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 -- 172.21.15.105:0/714110058 shutdown_connections 2024-03-23T07:56:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.956+0000 7f9f687e7740 1 -- 172.21.15.105:0/714110058 wait complete. 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.957+0000 7f9f687e7740 1 -- start start 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.957+0000 7f9f687e7740 1 Processor -- start 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 0x55a87bde5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bde6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f687e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bda2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f687e7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bda3bf0 con 0x55a87bd690e0 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87bd7c840 con 0x55a87bd68d00 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f687e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a87bd50d30 con 0x55a87bd525d0 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f687e7740 10 client.0 _ll_register_callbacks cb 0x7f9f48001d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f5f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bda2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f5f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bda2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47568/0 (socket says 172.21.15.105:47568) 2024-03-23T07:56:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f5f400640 1 -- 172.21.15.105:0/1314905333 learned_addr learned my addr 172.21.15.105:0/1314905333 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f57e00640 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bde6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.958+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 0x55a87bde5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f5f400640 1 -- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 msgr2=0x55a87bde5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f5f400640 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 0x55a87bde5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f5f400640 1 -- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 msgr2=0x55a87bde6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f5f400640 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bde6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f5f400640 1 -- 172.21.15.105:0/1314905333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a87bca3ff0 con 0x55a87bd690e0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f5f400640 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bda2d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f600ad770 tx=0x7f9f5800e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4622df3a1f8752 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f43e00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f43e00640 1 -- 172.21.15.105:0/1314905333 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58006b50 con 0x55a87bd690e0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f43e00640 1 -- 172.21.15.105:0/1314905333 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f58006cb0 con 0x55a87bd690e0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f5001a070 con 0x55a87bd690e0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f43e00640 1 -- 172.21.15.105:0/1314905333 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f58006e70 con 0x55a87bd690e0 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f687e7740 10 client.4797 fetch_fsmap learned FSMap version 8 2024-03-23T07:56:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.959+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a87bc462b0 con 0x55a87bd690e0 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f43e00640 1 -- 172.21.15.105:0/1314905333 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5001a070 con 0x55a87bd690e0 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f43e00640 1 -- 172.21.15.105:0/1314905333 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f9f5801b3a0 con 0x55a87bd690e0 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f687e7740 10 client.4797 fetch_fsmap finished waiting for FSMap version 8 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f687e7740 10 client.4797 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f687e7740 20 client.4797 populate_metadata read hostname 'smithi105' 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f687e7740 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bdd26c0 0x55a87bda3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f687e7740 4 client.4797 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 --> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a87b9aa6c0 con 0x55a87bdd26c0 2024-03-23T07:56:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.960+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bdd26c0 0x55a87bda3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:56:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.961+0000 7f9f5ea00640 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bdd26c0 0x55a87bda3280 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390311a98ced9546 server_cookie=2f43f0316d8d7697 in_seq=0 out_seq=0 2024-03-23T07:56:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.961+0000 7f9f43e00640 10 client.4797 ms_handle_connect on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.962+0000 7f9f43e00640 1 -- 172.21.15.105:0/1314905333 <== mds.0 v2:172.21.15.132:6832/713904573 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55a87b9aa6c0 con 0x55a87bdd26c0 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.962+0000 7f9f43e00640 10 client.4797 handle_command_reply: tid=0 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.962+0000 7f9f687e7740 1 client.4797 shutdown 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.962+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 msgr2=0x55a87bda2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.962+0000 7f9f687e7740 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bda2d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f600ad770 tx=0x7f9f5800e5e0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 shutdown_connections 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:6832/713904573,v1:172.21.15.132:6834/713904573] conn(0x55a87bdd26c0 0x55a87bda3280 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87bd690e0 0x55a87bda2d40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a87bd68d00 0x55a87bde6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f43e00640 0 client.4797 ms_handle_reset on v2:172.21.15.132:6832/713904573 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 --2- 172.21.15.105:0/1314905333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a87bd525d0 0x55a87bde5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 >> 172.21.15.105:0/1314905333 conn(0x55a87bd3c1e0 msgr2=0x55a87bd639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 shutdown_connections 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 1 -- 172.21.15.105:0/1314905333 wait complete. 2024-03-23T07:56:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f687e7740 20 client.4797 trim_cache size 0 max 16384 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "e641b836-a0e5-4e95-93aa-bb0628868335", 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4282, 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 42.261358583000003, 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 8, 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 27, 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 26, 2024-03-23T07:56:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 46.920153943000003 2024-03-23T07:56:59.967 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:56:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.963+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f3c06ac30 msgr2=0x7f9f3c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f3c06ac30 0x7f9f3c06d0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f60074b10 tx=0x7f9f50008040 comp rx=0 tx=0).stop 2024-03-23T07:56:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6014d2d0 msgr2=0x7f9f601f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:56:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6014d2d0 0x7f9f601f66f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800c280 tx=0x7f9f4802e070 comp rx=0 tx=0).stop 2024-03-23T07:56:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 shutdown_connections 2024-03-23T07:56:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f601f6c30 0x7f9f601f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6014d2d0 0x7f9f601f66f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9f3c06ac30 0x7f9f3c06d0f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 --2- 172.21.15.105:0/3324162897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f440047c0 0x7f9f6014cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:56:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 >> 172.21.15.105:0/3324162897 conn(0x7f9f6006a0b0 msgr2=0x7f9f601565d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:56:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 shutdown_connections 2024-03-23T07:56:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:56:59.964+0000 7f9f65a00640 1 -- 172.21.15.105:0/3324162897 wait complete. 2024-03-23T07:56:59.982 INFO:teuthology.run_tasks:Running task kclient... 2024-03-23T07:56:59.996 INFO:tasks.kclient:Mounting kernel clients... 2024-03-23T07:56:59.996 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-23T07:56:59.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:56:59.997 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:57:00.043 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-23T07:57:00.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.044 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-23T07:57:00.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.177 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-23T07:57:00.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.307 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:57:00.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.348 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-23T07:57:00.484 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-03-23T07:57:00.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:57:00.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.488 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-23T07:57:00.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.543 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-03-23T07:57:00.640 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-03-23T07:57:00.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:57:00.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.646 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-23T07:57:00.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:00.701 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-03-23T07:57:00.795 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-03-23T07:57:00.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:57:00.799 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-23T07:57:00.800 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T07:57:00.800 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T07:57:00.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T07:57:01.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.018+0000 7feebb200640 1 Processor -- start 2024-03-23T07:57:01.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.018+0000 7feebb200640 1 -- start start 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.019+0000 7feebb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 0x7feeb415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.019+0000 7feebb200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feeb415d410 0x7feeb40b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feebb200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb406c6f0 con 0x7feeb40b39b0 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feebb200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb40b4370 con 0x7feeb415d410 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feebb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb406c9d0 con 0x7feeb415b560 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feeb3400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feeb40b39b0 0x7feeb415e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33982/0 (socket says 172.21.15.105:33982) 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feeb3400640 1 -- 172.21.15.105:0/1380616399 learned_addr learned my addr 172.21.15.105:0/1380616399 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feeb3e00640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 0x7feeb415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feeb9200640 1 --2- 172.21.15.105:0/1380616399 >> v2:172.21.15.132:3300/0 conn(0x7feeb415d410 0x7feeb40b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251250873 0 0) 0x7feeb406c6f0 con 0x7feeb40b39b0 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.020+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee98003520 con 0x7feeb40b39b0 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 745515611 0 0) 0x7fee98003520 con 0x7feeb40b39b0 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 msgr2=0x7feeb415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 0x7feeb415b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 >> v2:172.21.15.132:3300/0 conn(0x7feeb415d410 msgr2=0x7feeb40b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 --2- 172.21.15.105:0/1380616399 >> v2:172.21.15.132:3300/0 conn(0x7feeb415d410 0x7feeb40b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb406ccb0 con 0x7feeb40b39b0 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7feea0001670 con 0x7feeb40b39b0 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 >> v1:172.21.15.105:6789/0 conn(0x7feeb40b39b0 legacy=0x7feeb415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee98003970 0x7fee98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 0x7fee980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee98004d00 0x7fee98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40b4370 con 0x7fee98004d00 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb406c9d0 con 0x7fee98003970 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9800b980 con 0x7feeb415b560 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb9200640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee98004d00 0x7fee98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.021+0000 7feeb3400640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 0x7fee980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.022+0000 7feeb3e00640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee98003970 0x7fee98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.022+0000 7feeb9200640 1 -- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 msgr2=0x7fee980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.022+0000 7feeb9200640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feeb415b560 0x7fee980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.022+0000 7feeb9200640 1 -- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee98003970 msgr2=0x7fee98003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.022+0000 7feeb9200640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee98003970 0x7fee98003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.022+0000 7feeb9200640 1 -- 172.21.15.105:0/1380616399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98003520 con 0x7fee98004d00 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.022+0000 7feeb3e00640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee98003970 0x7fee98003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:01.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.023+0000 7feeb9200640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee98004d00 0x7fee98005180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feeac009f90 tx=0x7fee980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31c283eaebf6912 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.023+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac02cc20 con 0x7fee98004d00 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.023+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac00f040 con 0x7fee98004d00 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.023+0000 7feeb2a00640 1 -- 172.21.15.105:0/1380616399 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac034440 con 0x7fee98004d00 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.023+0000 7feebb200640 1 -- 172.21.15.105:0/1380616399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee98004d00 msgr2=0x7fee98005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.023+0000 7feebb200640 1 --2- 172.21.15.105:0/1380616399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee98004d00 0x7fee98005180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feeac009f90 tx=0x7fee980057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.024+0000 7feeb9200640 1 -- 172.21.15.105:0/1380616399 reap_dead start 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.024+0000 7feebb200640 1 -- 172.21.15.105:0/1380616399 shutdown_connections 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.024+0000 7feebb200640 1 -- 172.21.15.105:0/1380616399 >> 172.21.15.105:0/1380616399 conn(0x7feeb406a3d0 msgr2=0x7feeb40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.024+0000 7feebb200640 1 -- 172.21.15.105:0/1380616399 shutdown_connections 2024-03-23T07:57:01.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.024+0000 7feebb200640 1 -- 172.21.15.105:0/1380616399 wait complete. 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.026+0000 7feebb200640 1 Processor -- start 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.026+0000 7feebb200640 1 -- start start 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feebb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee98004d00 0x7feeb413c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feebb200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb4139850 0x7feeb41efd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feebb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb4153450 0x7feeb41f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feebb200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb41f0270 con 0x7feeb4139850 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feebb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb415aab0 con 0x7feeb4153450 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feebb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb413a690 con 0x7fee98004d00 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb4139850 0x7feeb41efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb4139850 0x7feeb41efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47590/0 (socket says 172.21.15.105:47590) 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 -- 172.21.15.105:0/2389913237 learned_addr learned my addr 172.21.15.105:0/2389913237 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:01.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3e00640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee98004d00 0x7feeb413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb9200640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb4153450 0x7feeb41f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 -- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee98004d00 msgr2=0x7feeb413c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee98004d00 0x7feeb413c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 -- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb4153450 msgr2=0x7feeb41f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb4153450 0x7feeb41f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3400640 1 -- 172.21.15.105:0/2389913237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb4139df0 con 0x7feeb4139850 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.027+0000 7feeb3e00640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee98004d00 0x7feeb413c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.028+0000 7feeb3400640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb4139850 0x7feeb41efd30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feea0001eb0 tx=0x7feea000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.028+0000 7feeabe00640 1 -- 172.21.15.105:0/2389913237 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea000c8d0 con 0x7feeb4139850 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.028+0000 7feeabe00640 1 -- 172.21.15.105:0/2389913237 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea000ca50 con 0x7feeb4139850 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.028+0000 7feeabe00640 1 -- 172.21.15.105:0/2389913237 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea000ce00 con 0x7feeb4139850 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.028+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb413a0d0 con 0x7feeb4139850 2024-03-23T07:57:01.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.028+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb41f3950 con 0x7feeb4139850 2024-03-23T07:57:01.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.029+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb406ac20 con 0x7feeb4139850 2024-03-23T07:57:01.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.030+0000 7feeabe00640 1 -- 172.21.15.105:0/2389913237 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feea001d460 con 0x7feeb4139850 2024-03-23T07:57:01.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.031+0000 7feeabe00640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fee9406ac00 0x7fee9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:01.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.031+0000 7feeabe00640 1 -- 172.21.15.105:0/2389913237 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7feea0089690 con 0x7feeb4139850 2024-03-23T07:57:01.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.031+0000 7feeb3e00640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fee9406ac00 0x7fee9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:01.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.032+0000 7feeb3e00640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fee9406ac00 0x7fee9406d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feea4009aa0 tx=0x7feea4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:01.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.035+0000 7feeabe00640 1 -- 172.21.15.105:0/2389913237 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feea0054590 con 0x7feeb4139850 2024-03-23T07:57:01.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.271+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feeb4073390 con 0x7feeb4139850 2024-03-23T07:57:01.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.273+0000 7feeabe00640 1 -- 172.21.15.105:0/2389913237 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7feea0057bb0 con 0x7feeb4139850 2024-03-23T07:57:01.273 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-23T07:57:01.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.274+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fee9406ac00 msgr2=0x7fee9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:01.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.274+0000 7feebb200640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fee9406ac00 0x7fee9406d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feea4009aa0 tx=0x7feea4008040 comp rx=0 tx=0).stop 2024-03-23T07:57:01.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.274+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb4139850 msgr2=0x7feeb41efd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:01.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.274+0000 7feebb200640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb4139850 0x7feeb41efd30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feea0001eb0 tx=0x7feea000ec80 comp rx=0 tx=0).stop 2024-03-23T07:57:01.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.274+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 shutdown_connections 2024-03-23T07:57:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.275+0000 7feebb200640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb4153450 0x7feeb41f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.275+0000 7feebb200640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feeb4139850 0x7feeb41efd30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.275+0000 7feebb200640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fee9406ac00 0x7fee9406d0c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.275+0000 7feebb200640 1 --2- 172.21.15.105:0/2389913237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee98004d00 0x7feeb413c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.275+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 >> 172.21.15.105:0/2389913237 conn(0x7feeb406a3d0 msgr2=0x7feeb415d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.275+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 shutdown_connections 2024-03-23T07:57:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:01.275+0000 7feebb200640 1 -- 172.21.15.105:0/2389913237 wait complete. 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T07:57:01.294 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T07:57:01.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.295 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-23T07:57:01.344 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T07:57:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42504sec preferred_lft 42504sec 2024-03-23T07:57:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-23T07:57:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:01.347 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T07:57:01.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.347 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:01.347 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:01.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-03-23T07:57:01.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-03-23T07:57:01.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-03-23T07:57:01.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T07:57:01.347 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:01.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.559 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T07:57:01.637 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-23T07:57:01.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.641 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-23T07:57:01.690 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-23T07:57:01.690 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T07:57:01.690 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T07:57:01.690 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T07:57:01.690 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T07:57:01.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.691 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:01.691 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:01.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T07:57:01.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T07:57:01.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T07:57:01.692 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:01.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.872 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:57:01.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.922 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-03-23T07:57:01.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:01.972 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:01.972 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:01.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T07:57:01.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T07:57:01.973 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:02.109 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T07:57:02.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T07:57:02.110 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:02.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:02.347 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:02.347 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:02.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-03-23T07:57:02.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T07:57:02.347 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:02.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:02.501 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T07:57:02.548 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T07:57:02.548 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:02.548 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T07:57:02.642 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-23T07:57:02.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:02.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:02.779 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:02.779 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:03.434 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:03.434 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:03.434 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-03-23T07:57:03.434 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 0@e641b836-a0e5-4e95-93aa-bb0628868335.cephfs=/ 2024-03-23T07:57:03.434 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.105:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2024-03-23T07:57:03.436 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-23T07:57:03.437 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-03-23T07:57:03.437 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T07:57:03.437 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T07:57:03.437 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T07:57:03.440 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T07:57:03.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:03.441 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T07:57:03.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:03.522 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:03.578 INFO:teuthology.orchestra.run.smithi105.stdout:client4630 2024-03-23T07:57:03.578 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:03.578 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:03.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:03.630 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:03.731 INFO:teuthology.orchestra.run.smithi105.stdout:client4630 2024-03-23T07:57:03.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:03.731 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:client4630 2024-03-23T07:57:03.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:03.786 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e641b836-a0e5-4e95-93aa-bb0628868335.client4630/status') 2024-03-23T07:57:03.872 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4630 (3)172.21.15.105:0/2538460897 2024-03-23T07:57:03.873 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-03-23T07:57:03.873 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-03-23T07:57:03.873 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-03-23T07:57:03.873 INFO:teuthology.orchestra.run.smithi105.stderr:71 bytes copied, 6.1885e-05 s, 1.1 MB/s 2024-03-23T07:57:03.877 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-23T07:57:03.877 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T07:57:03.877 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-23T07:57:03.878 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T07:57:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.083+0000 7f44ea600640 1 Processor -- start 2024-03-23T07:57:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.084+0000 7f44ea600640 1 -- start start 2024-03-23T07:57:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44ea600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44e415cda0 0x7f44e40cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44ea600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b4a60 0x7f44e40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44ea600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40d1830 con 0x7f44e40b5a30 2024-03-23T07:57:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44ea600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e40b1f20 con 0x7f44e415cda0 2024-03-23T07:57:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44ea600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40d0c20 con 0x7f44e40b4a60 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44e3400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44e415cda0 0x7f44e40cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44e3400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44e415cda0 0x7f44e40cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47594/0 (socket says 172.21.15.105:47594) 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.085+0000 7f44e3400640 1 -- 172.21.15.105:0/3713840131 learned_addr learned my addr 172.21.15.105:0/3713840131 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.086+0000 7f44e3e00640 1 --2- 172.21.15.105:0/3713840131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b4a60 0x7f44e40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.086+0000 7f44e3400640 1 -- 172.21.15.105:0/3713840131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b4a60 msgr2=0x7f44e40b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.086+0000 7f44e3400640 1 --2- 172.21.15.105:0/3713840131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b4a60 0x7f44e40b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.086+0000 7f44e3400640 1 -- 172.21.15.105:0/3713840131 >> v1:172.21.15.132:6789/0 conn(0x7f44e40b5a30 legacy=0x7f44e40b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.086+0000 7f44e3400640 1 -- 172.21.15.105:0/3713840131 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40d0f00 con 0x7f44e415cda0 2024-03-23T07:57:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.087+0000 7f44e3400640 1 --2- 172.21.15.105:0/3713840131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e415cda0 0x7f44e40cdb40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009e50 tx=0x7f44d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9eb8c4426e119bc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.087+0000 7f44e2000640 1 -- 172.21.15.105:0/3713840131 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d402ca70 con 0x7f44e415cda0 2024-03-23T07:57:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.087+0000 7f44e2000640 1 -- 172.21.15.105:0/3713840131 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d402cbd0 con 0x7f44e415cda0 2024-03-23T07:57:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.087+0000 7f44e2000640 1 -- 172.21.15.105:0/3713840131 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40343f0 con 0x7f44e415cda0 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 -- 172.21.15.105:0/3713840131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e415cda0 msgr2=0x7f44e40cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 --2- 172.21.15.105:0/3713840131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e415cda0 0x7f44e40cdb40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009e50 tx=0x7f44d402bed0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 -- 172.21.15.105:0/3713840131 shutdown_connections 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 --2- 172.21.15.105:0/3713840131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b4a60 0x7f44e40b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 --2- 172.21.15.105:0/3713840131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e415cda0 0x7f44e40cdb40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 -- 172.21.15.105:0/3713840131 >> 172.21.15.105:0/3713840131 conn(0x7f44e40cf350 msgr2=0x7f44e40cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 -- 172.21.15.105:0/3713840131 shutdown_connections 2024-03-23T07:57:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.088+0000 7f44ea600640 1 -- 172.21.15.105:0/3713840131 wait complete. 2024-03-23T07:57:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.090+0000 7f44ea600640 1 Processor -- start 2024-03-23T07:57:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.090+0000 7f44ea600640 1 -- start start 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44ea600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e40b4a60 0x7f44e40c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44ea600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b5a30 0x7f44e40d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44ea600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e415cda0 0x7f44e414ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44ea600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40c9810 con 0x7f44e40b4a60 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44ea600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40c8c00 con 0x7f44e415cda0 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44ea600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40c8ee0 con 0x7f44e40b5a30 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44e3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e40b4a60 0x7f44e40c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44e3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e40b4a60 0x7f44e40c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47596/0 (socket says 172.21.15.105:47596) 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44e3400640 1 -- 172.21.15.105:0/397320827 learned_addr learned my addr 172.21.15.105:0/397320827 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44e3400640 1 -- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b5a30 msgr2=0x7f44e40d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.091+0000 7f44e2a00640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b5a30 0x7f44e40d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44e3400640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b5a30 0x7f44e40d1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44e3e00640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e415cda0 0x7f44e414ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44e3400640 1 -- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e415cda0 msgr2=0x7f44e414ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44e3400640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e415cda0 0x7f44e414ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44e3400640 1 -- 172.21.15.105:0/397320827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009b30 con 0x7f44e40b4a60 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44e3400640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e40b4a60 0x7f44e40c8310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009e50 tx=0x7f44d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44db400640 1 -- 172.21.15.105:0/397320827 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4009510 con 0x7f44e40b4a60 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44db400640 1 -- 172.21.15.105:0/397320827 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d403e450 con 0x7f44e40b4a60 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e40c91c0 con 0x7f44e40b4a60 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44db400640 1 -- 172.21.15.105:0/397320827 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d403e800 con 0x7f44e40b4a60 2024-03-23T07:57:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.092+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e40c8850 con 0x7f44e40b4a60 2024-03-23T07:57:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.094+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e4073390 con 0x7f44e40b4a60 2024-03-23T07:57:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.094+0000 7f44db400640 1 -- 172.21.15.105:0/397320827 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f44d403e960 con 0x7f44e40b4a60 2024-03-23T07:57:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.095+0000 7f44db400640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f44c006ac30 0x7f44c006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.095+0000 7f44db400640 1 -- 172.21.15.105:0/397320827 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f44d40ac260 con 0x7f44e40b4a60 2024-03-23T07:57:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.095+0000 7f44e2a00640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f44c006ac30 0x7f44c006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.096+0000 7f44e2a00640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f44c006ac30 0x7f44c006d0f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f44e41f5720 tx=0x7f44cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.100+0000 7f44db400640 1 -- 172.21.15.105:0/397320827 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f44d4077050 con 0x7f44e40b4a60 2024-03-23T07:57:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.335+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f44e40c9020 con 0x7f44e40b4a60 2024-03-23T07:57:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.336+0000 7f44db400640 1 -- 172.21.15.105:0/397320827 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f44e40c9020 con 0x7f44e40b4a60 2024-03-23T07:57:04.335 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-23T07:57:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f44c006ac30 msgr2=0x7f44c006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f44c006ac30 0x7f44c006d0f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f44e41f5720 tx=0x7f44cc009040 comp rx=0 tx=0).stop 2024-03-23T07:57:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e40b4a60 msgr2=0x7f44e40c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e40b4a60 0x7f44e40c8310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009e50 tx=0x7f44d4004a40 comp rx=0 tx=0).stop 2024-03-23T07:57:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 shutdown_connections 2024-03-23T07:57:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e415cda0 0x7f44e414ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f44c006ac30 0x7f44c006d0f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e40b5a30 0x7f44e40d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 --2- 172.21.15.105:0/397320827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44e40b4a60 0x7f44e40c8310 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.338+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 >> 172.21.15.105:0/397320827 conn(0x7f44e40cf350 msgr2=0x7f44e40b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.339+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 shutdown_connections 2024-03-23T07:57:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:04.339+0000 7f44ea600640 1 -- 172.21.15.105:0/397320827 wait complete. 2024-03-23T07:57:04.355 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T07:57:04.355 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T07:57:04.355 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-23T07:57:04.355 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-23T07:57:04.355 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-23T07:57:04.355 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T07:57:04.355 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T07:57:04.356 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T07:57:04.356 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-23T07:57:04.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.356 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:04.405 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42501sec preferred_lft 42501sec 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether de:d6:9d:2d:cf:cd brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:04.406 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::88e7:17ff:fe0f:aec1/64 scope link 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run.smithi105.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether de:d6:9d:2d:cf:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::dcd6:9dff:fe2d:cfcd/64 scope link 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:04.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.407 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T07:57:04.533 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-23T07:57:04.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.537 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-23T07:57:04.587 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-23T07:57:04.587 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T07:57:04.587 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T07:57:04.587 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T07:57:04.587 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T07:57:04.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.588 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:04.588 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:04.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T07:57:04.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T07:57:04.589 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T07:57:04.589 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:04.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.759 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:57:04.807 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T07:57:04.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.808 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-03-23T07:57:04.858 INFO:teuthology.orchestra.run.smithi105.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-23T07:57:04.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.859 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:04.859 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:04.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T07:57:04.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-23T07:57:04.859 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:04.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:04.997 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 96:1d:2c:77:68:3b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::941d:2cff:fe77:683b/64 scope link 2024-03-23T07:57:05.092 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:05.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 96:1d:2c:77:68:3b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::941d:2cff:fe77:683b/64 scope link 2024-03-23T07:57:05.187 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:05.192 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-23T07:57:05.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:05.192 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:05.192 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:05.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-23T07:57:05.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-23T07:57:05.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-23T07:57:05.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-23T07:57:05.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-23T07:57:05.193 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:05.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:05.454 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:57:05.455 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:57:05.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2024-03-23T07:57:05.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.1 master ceph-brx 2024-03-23T07:57:05.455 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:57:05.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:05.623 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-23T07:57:05.668 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-23T07:57:05.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:05.668 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-23T07:57:05.764 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-23T07:57:05.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:05.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:05.894 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:05.894 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:05.930 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:05.930 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:05.930 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-03-23T07:57:05.930 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 2@e641b836-a0e5-4e95-93aa-bb0628868335.cephfs=/ 2024-03-23T07:57:05.930 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.105:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2024-03-23T07:57:05.932 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-23T07:57:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-03-23T07:57:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T07:57:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T07:57:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T07:57:05.936 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T07:57:05.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:05.937 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-23T07:57:06.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:06.019 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:06.074 INFO:teuthology.orchestra.run.smithi105.stdout:client4493 2024-03-23T07:57:06.074 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:06.075 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:06.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:06.125 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:06.179 INFO:teuthology.orchestra.run.smithi105.stdout:client4493 2024-03-23T07:57:06.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:06.180 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:06.234 INFO:teuthology.orchestra.run.smithi105.stdout:client4493 2024-03-23T07:57:06.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:06.235 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e641b836-a0e5-4e95-93aa-bb0628868335.client4493/status') 2024-03-23T07:57:06.323 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4493 (3)172.21.15.105:0/3480601004 2024-03-23T07:57:06.323 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-03-23T07:57:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-03-23T07:57:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-03-23T07:57:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:71 bytes copied, 7.3765e-05 s, 963 kB/s 2024-03-23T07:57:06.327 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-23T07:57:06.327 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T07:57:06.327 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-23T07:57:06.328 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T07:57:06.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.529+0000 7fead4000640 1 Processor -- start 2024-03-23T07:57:06.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.530+0000 7fead4000640 1 -- start start 2024-03-23T07:57:06.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.531+0000 7fead4000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 0x7feacc098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.531+0000 7fead4000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc0cb6b0 con 0x7feacc15a950 2024-03-23T07:57:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.531+0000 7fead4000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc15ad30 con 0x7feacc098f80 2024-03-23T07:57:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.531+0000 7fead4000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc099b40 con 0x7feacc0cb250 2024-03-23T07:57:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.531+0000 7fead1600640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feacc15a950 0x7feacc0cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33986/0 (socket says 172.21.15.105:33986) 2024-03-23T07:57:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.531+0000 7fead1600640 1 -- 172.21.15.105:0/2922217156 learned_addr learned my addr 172.21.15.105:0/2922217156 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7fead0c00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 0x7feacc098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935473199 0 0) 0x7feacc0cb6b0 con 0x7feacc15a950 2024-03-23T07:57:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab0003520 con 0x7feacc15a950 2024-03-23T07:57:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2089204720 0 0) 0x7feab0003520 con 0x7feacc15a950 2024-03-23T07:57:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 msgr2=0x7feacc098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 0x7feacc098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 >> v1:172.21.15.132:6789/0 conn(0x7feacc098f80 legacy=0x7feacc0b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0cb930 con 0x7feacc15a950 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7feabc001670 con 0x7feacc15a950 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 >> v1:172.21.15.105:6789/0 conn(0x7feacc15a950 legacy=0x7feacc0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00039a0 0x7feab0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab00047c0 0x7feab0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.532+0000 7feacbe00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 0x7feab0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab000bb10 con 0x7feab00047c0 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab000bc50 con 0x7feab00039a0 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab000be10 con 0x7feacc0cb250 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead0c00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab00047c0 0x7feab0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:06.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead1600640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00039a0 0x7feab0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead2000640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 0x7feab0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead0c00640 1 -- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 msgr2=0x7feab0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead0c00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 0x7feab0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead0c00640 1 -- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00039a0 msgr2=0x7feab0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead0c00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00039a0 0x7feab0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead0c00640 1 -- 172.21.15.105:0/2922217156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0003520 con 0x7feab00047c0 2024-03-23T07:57:06.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.533+0000 7fead2000640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc0cb250 0x7feab0005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.534+0000 7fead0c00640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab00047c0 0x7feab0004c40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feab8009ac0 tx=0x7feab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eec7ff115d60218 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab802c6b0 con 0x7feab00047c0 2024-03-23T07:57:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab802c810 con 0x7feab00047c0 2024-03-23T07:57:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7feacbe00640 1 -- 172.21.15.105:0/2922217156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab802cba0 con 0x7feab00047c0 2024-03-23T07:57:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7fead4000640 1 -- 172.21.15.105:0/2922217156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab00047c0 msgr2=0x7feab0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7fead4000640 1 --2- 172.21.15.105:0/2922217156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab00047c0 0x7feab0004c40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feab8009ac0 tx=0x7feab00057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7fead2000640 1 -- 172.21.15.105:0/2922217156 reap_dead start 2024-03-23T07:57:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7fead4000640 1 -- 172.21.15.105:0/2922217156 shutdown_connections 2024-03-23T07:57:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7fead4000640 1 -- 172.21.15.105:0/2922217156 >> 172.21.15.105:0/2922217156 conn(0x7feacc0b21a0 msgr2=0x7feacc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7fead4000640 1 -- 172.21.15.105:0/2922217156 shutdown_connections 2024-03-23T07:57:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.535+0000 7fead4000640 1 -- 172.21.15.105:0/2922217156 wait complete. 2024-03-23T07:57:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.537+0000 7fead4000640 1 Processor -- start 2024-03-23T07:57:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.537+0000 7fead4000640 1 -- start start 2024-03-23T07:57:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.538+0000 7fead4000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00047c0 0x7feacc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.538+0000 7fead4000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc155a10 0x7feacc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.538+0000 7fead4000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feacc15d420 0x7feacc15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.538+0000 7fead4000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc15dde0 con 0x7feacc15d420 2024-03-23T07:57:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.538+0000 7fead4000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0c97b0 con 0x7feab00047c0 2024-03-23T07:57:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.538+0000 7fead4000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0c9a90 con 0x7feacc155a10 2024-03-23T07:57:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.538+0000 7fead2000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feacc15d420 0x7feacc15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feacc15d420 0x7feacc15d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47610/0 (socket says 172.21.15.105:47610) 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 -- 172.21.15.105:0/3041733871 learned_addr learned my addr 172.21.15.105:0/3041733871 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 -- 172.21.15.105:0/3041733871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc155a10 msgr2=0x7feacc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc155a10 0x7feacc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 -- 172.21.15.105:0/3041733871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00047c0 msgr2=0x7feacc155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00047c0 0x7feacc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 -- 172.21.15.105:0/3041733871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0c9d70 con 0x7feacc15d420 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead2000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feacc15d420 0x7feacc15d8a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feac4000f80 tx=0x7feac400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7feac9600640 1 -- 172.21.15.105:0/3041733871 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4017070 con 0x7feacc15d420 2024-03-23T07:57:06.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7feac9600640 1 -- 172.21.15.105:0/3041733871 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac4006b50 con 0x7feacc15d420 2024-03-23T07:57:06.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc0ca050 con 0x7feacc15d420 2024-03-23T07:57:06.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.539+0000 7feac9600640 1 -- 172.21.15.105:0/3041733871 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4003970 con 0x7feacc15d420 2024-03-23T07:57:06.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.540+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc06cb30 con 0x7feacc15d420 2024-03-23T07:57:06.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.541+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc0ca1e0 con 0x7feacc15d420 2024-03-23T07:57:06.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.542+0000 7feac9600640 1 -- 172.21.15.105:0/3041733871 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feac4003ad0 con 0x7feacc15d420 2024-03-23T07:57:06.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.543+0000 7feac9600640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feaac06ac30 0x7feaac06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:06.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.543+0000 7feac9600640 1 -- 172.21.15.105:0/3041733871 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7feac4088f10 con 0x7feacc15d420 2024-03-23T07:57:06.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.543+0000 7fead1600640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feaac06ac30 0x7feaac06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:06.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.544+0000 7fead1600640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feaac06ac30 0x7feaac06d0f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feabc001050 tx=0x7feabc000f40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:06.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.547+0000 7feac9600640 1 -- 172.21.15.105:0/3041733871 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feac4053dd0 con 0x7feacc15d420 2024-03-23T07:57:06.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.785+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feacc066bb0 con 0x7feacc15d420 2024-03-23T07:57:06.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.786+0000 7feac9600640 1 -- 172.21.15.105:0/3041733871 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7feac40573f0 con 0x7feacc15d420 2024-03-23T07:57:06.785 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-23T07:57:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feaac06ac30 msgr2=0x7feaac06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feaac06ac30 0x7feaac06d0f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feabc001050 tx=0x7feabc000f40 comp rx=0 tx=0).stop 2024-03-23T07:57:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feacc15d420 msgr2=0x7feacc15d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feacc15d420 0x7feacc15d8a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feac4000f80 tx=0x7feac400dc00 comp rx=0 tx=0).stop 2024-03-23T07:57:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 shutdown_connections 2024-03-23T07:57:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feacc15d420 0x7feacc15d8a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feaac06ac30 0x7feaac06d0f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feacc155a10 0x7feacc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 --2- 172.21.15.105:0/3041733871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feab00047c0 0x7feacc155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.788+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 >> 172.21.15.105:0/3041733871 conn(0x7feacc0b21a0 msgr2=0x7feacc0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.789+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 shutdown_connections 2024-03-23T07:57:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:06.789+0000 7fead4000640 1 -- 172.21.15.105:0/3041733871 wait complete. 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T07:57:06.807 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-23T07:57:06.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:06.808 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:06.864 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42520sec preferred_lft 42520sec 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-03-23T07:57:06.865 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:06.866 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T07:57:06.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:06.866 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:06.866 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:06.866 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2024-03-23T07:57:06.866 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2024-03-23T07:57:06.866 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2024-03-23T07:57:06.866 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T07:57:06.867 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:07.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:07.310 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T07:57:07.380 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-03-23T07:57:07.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:07.384 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-03-23T07:57:07.434 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-03-23T07:57:07.434 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T07:57:07.434 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T07:57:07.434 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T07:57:07.434 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T07:57:07.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:07.435 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:07.435 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:07.435 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T07:57:07.435 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T07:57:07.435 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T07:57:07.435 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:08.019 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:57:08.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:57:08.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:08.161 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-23T07:57:08.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:08.247 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-03-23T07:57:08.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:08.337 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:08.337 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:08.337 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-23T07:57:08.337 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-23T07:57:08.337 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:08.458 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-23T07:57:08.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-23T07:57:08.458 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:08.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:08.726 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:08.726 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:08.726 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-03-23T07:57:08.727 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T07:57:08.727 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:08.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:08.874 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-23T07:57:08.919 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-23T07:57:08.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:08.920 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-23T07:57:09.011 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-23T07:57:09.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:09.012 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:09.146 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:09.146 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:09.848 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:09.849 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:09.849 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-03-23T07:57:09.849 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with new device syntax: 1@e641b836-a0e5-4e95-93aa-bb0628868335.cephfs=/ 2024-03-23T07:57:09.849 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.105:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2024-03-23T07:57:09.853 INFO:teuthology.orchestra.run.smithi132.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-23T07:57:09.853 INFO:teuthology.orchestra.run.smithi132.stdout: You just mounted a file system that supports labels which does not 2024-03-23T07:57:09.853 INFO:teuthology.orchestra.run.smithi132.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T07:57:09.853 INFO:teuthology.orchestra.run.smithi132.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T07:57:09.853 INFO:teuthology.orchestra.run.smithi132.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T07:57:09.858 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T07:57:09.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:09.859 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-23T07:57:09.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:09.951 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:10.008 INFO:teuthology.orchestra.run.smithi132.stdout:client4636 2024-03-23T07:57:10.008 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:10.008 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:10.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:10.058 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:10.106 INFO:teuthology.orchestra.run.smithi132.stdout:client4636 2024-03-23T07:57:10.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:10.106 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:10.158 INFO:teuthology.orchestra.run.smithi132.stdout:client4636 2024-03-23T07:57:10.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:10.158 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e641b836-a0e5-4e95-93aa-bb0628868335.client4636/status') 2024-03-23T07:57:10.246 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.4636 (3)192.168.0.1:0/3096988848 2024-03-23T07:57:10.246 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2024-03-23T07:57:10.246 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2024-03-23T07:57:10.246 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2024-03-23T07:57:10.246 INFO:teuthology.orchestra.run.smithi132.stderr:69 bytes copied, 8.8038e-05 s, 784 kB/s 2024-03-23T07:57:10.250 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-23T07:57:10.250 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T07:57:10.250 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-23T07:57:10.250 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T07:57:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.468+0000 7fc67b200640 1 Processor -- start 2024-03-23T07:57:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.469+0000 7fc67b200640 1 -- start start 2024-03-23T07:57:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.470+0000 7fc67b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc67413c740 0x7fc67413cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.470+0000 7fc67b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc67413d2d0 0x7fc6740b0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.470+0000 7fc67b200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67415a9a0 con 0x7fc6740b08d0 2024-03-23T07:57:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.470+0000 7fc67b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67415ac80 con 0x7fc67413c740 2024-03-23T07:57:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.470+0000 7fc67b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67413cb20 con 0x7fc67413d2d0 2024-03-23T07:57:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc679200640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc6740b08d0 0x7fc6740b27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40468/0 (socket says 172.21.15.105:40468) 2024-03-23T07:57:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc679200640 1 -- 172.21.15.105:0/1886026757 learned_addr learned my addr 172.21.15.105:0/1886026757 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673e00640 1 --2- 172.21.15.105:0/1886026757 >> v2:172.21.15.132:3300/0 conn(0x7fc67413c740 0x7fc67413cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc66b400640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc67413d2d0 0x7fc6740b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229631501 0 0) 0x7fc67415a9a0 con 0x7fc6740b08d0 2024-03-23T07:57:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc658005730 con 0x7fc6740b08d0 2024-03-23T07:57:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3770552491 0 0) 0x7fc658005730 con 0x7fc6740b08d0 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc67413d2d0 msgr2=0x7fc6740b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc67413d2d0 0x7fc6740b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 >> v2:172.21.15.132:3300/0 conn(0x7fc67413c740 msgr2=0x7fc67413cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 --2- 172.21.15.105:0/1886026757 >> v2:172.21.15.132:3300/0 conn(0x7fc67413c740 0x7fc67413cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740af8b0 con 0x7fc6740b08d0 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.471+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fc66c0019f0 con 0x7fc6740b08d0 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 >> v1:172.21.15.105:6789/0 conn(0x7fc6740b08d0 legacy=0x7fc6740b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673400640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc658005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673400640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc67413d2d0 0x7fc6580069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673400640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc658006f10 0x7fc658007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67415ac80 con 0x7fc658005b80 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67413cb20 con 0x7fc658006f10 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65800db90 con 0x7fc67413d2d0 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673e00640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc658005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673e00640 1 -- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc67413d2d0 msgr2=0x7fc6580069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673e00640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc67413d2d0 0x7fc6580069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673e00640 1 -- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc658006f10 msgr2=0x7fc658007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673e00640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc658006f10 0x7fc658007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.472+0000 7fc673e00640 1 -- 172.21.15.105:0/1886026757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc658005730 con 0x7fc658005b80 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.473+0000 7fc673e00640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc658005fa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc664009af0 tx=0x7fc658007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839de9a85a38d51c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66402c710 con 0x7fc658005b80 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66402c870 con 0x7fc658005b80 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc673400640 1 -- 172.21.15.105:0/1886026757 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66402cc00 con 0x7fc658005b80 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc67b200640 1 -- 172.21.15.105:0/1886026757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 msgr2=0x7fc658005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc67b200640 1 --2- 172.21.15.105:0/1886026757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc658005fa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc664009af0 tx=0x7fc658007a00 comp rx=0 tx=0).stop 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc679200640 1 -- 172.21.15.105:0/1886026757 reap_dead start 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc67b200640 1 -- 172.21.15.105:0/1886026757 shutdown_connections 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc67b200640 1 -- 172.21.15.105:0/1886026757 >> 172.21.15.105:0/1886026757 conn(0x7fc674065e20 msgr2=0x7fc674069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc67b200640 1 -- 172.21.15.105:0/1886026757 shutdown_connections 2024-03-23T07:57:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.474+0000 7fc67b200640 1 -- 172.21.15.105:0/1886026757 wait complete. 2024-03-23T07:57:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.476+0000 7fc67b200640 1 Processor -- start 2024-03-23T07:57:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.476+0000 7fc67b200640 1 -- start start 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc67b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc67413bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc67b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc674155ca0 0x7fc67415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc67b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc67415c350 0x7fc6740c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc67b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67415c7d0 con 0x7fc658005b80 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc67b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740c9a00 con 0x7fc67415c350 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc67b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc674154ee0 con 0x7fc674155ca0 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc673e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc67413bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc673e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc67413bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59856/0 (socket says 172.21.15.105:59856) 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.477+0000 7fc673e00640 1 -- 172.21.15.105:0/267625165 learned_addr learned my addr 172.21.15.105:0/267625165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc673e00640 1 -- 172.21.15.105:0/267625165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc674155ca0 msgr2=0x7fc67415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc673e00640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc674155ca0 0x7fc67415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc673e00640 1 -- 172.21.15.105:0/267625165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc67415c350 msgr2=0x7fc6740c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc673e00640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc67415c350 0x7fc6740c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc673e00640 1 -- 172.21.15.105:0/267625165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6741551c0 con 0x7fc658005b80 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc673e00640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc67413bd40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc66402c2b0 tx=0x7fc664004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc670c00640 1 -- 172.21.15.105:0/267625165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664003810 con 0x7fc658005b80 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc670c00640 1 -- 172.21.15.105:0/267625165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66402de90 con 0x7fc658005b80 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc670c00640 1 -- 172.21.15.105:0/267625165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664003b70 con 0x7fc658005b80 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.478+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6741554a0 con 0x7fc658005b80 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.479+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67413c280 con 0x7fc658005b80 2024-03-23T07:57:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.480+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc674073390 con 0x7fc658005b80 2024-03-23T07:57:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.480+0000 7fc670c00640 1 -- 172.21.15.105:0/267625165 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc664003cd0 con 0x7fc658005b80 2024-03-23T07:57:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.481+0000 7fc670c00640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc65006ac30 0x7fc65006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.481+0000 7fc670c00640 1 -- 172.21.15.105:0/267625165 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc6640ab620 con 0x7fc658005b80 2024-03-23T07:57:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.482+0000 7fc66b400640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc65006ac30 0x7fc65006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.482+0000 7fc66b400640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc65006ac30 0x7fc65006d0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc65c006a90 tx=0x7fc65c006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.486+0000 7fc670c00640 1 -- 172.21.15.105:0/267625165 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc664076380 con 0x7fc658005b80 2024-03-23T07:57:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.721+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc67406b250 con 0x7fc658005b80 2024-03-23T07:57:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.722+0000 7fc670c00640 1 -- 172.21.15.105:0/267625165 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc6640799a0 con 0x7fc658005b80 2024-03-23T07:57:10.722 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc65006ac30 msgr2=0x7fc65006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc65006ac30 0x7fc65006d0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc65c006a90 tx=0x7fc65c006a20 comp rx=0 tx=0).stop 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 msgr2=0x7fc67413bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc67413bd40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc66402c2b0 tx=0x7fc664004730 comp rx=0 tx=0).stop 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 shutdown_connections 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc67415c350 0x7fc6740c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc65006ac30 0x7fc65006d0f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc674155ca0 0x7fc67415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 --2- 172.21.15.105:0/267625165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc658005b80 0x7fc67413bd40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 >> 172.21.15.105:0/267625165 conn(0x7fc674065e20 msgr2=0x7fc67413d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 shutdown_connections 2024-03-23T07:57:10.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:10.725+0000 7fc67b200640 1 -- 172.21.15.105:0/267625165 wait complete. 2024-03-23T07:57:10.744 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T07:57:10.744 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T07:57:10.744 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-03-23T07:57:10.744 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-23T07:57:10.744 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-23T07:57:10.745 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T07:57:10.745 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T07:57:10.745 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T07:57:10.745 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-23T07:57:10.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:10.745 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42516sec preferred_lft 42516sec 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether f6:f4:0b:8e:ee:aa brd ff:ff:ff:ff:ff:ff 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::3c4f:39ff:fe7c:d75c/64 scope link 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:10.804 INFO:teuthology.orchestra.run.smithi132.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T07:57:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether f6:f4:0b:8e:ee:aa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-23T07:57:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::f4f4:bff:fe8e:eeaa/64 scope link 2024-03-23T07:57:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:10.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:10.805 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T07:57:10.896 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-03-23T07:57:10.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:10.901 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-03-23T07:57:10.954 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-03-23T07:57:10.954 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T07:57:10.954 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T07:57:10.954 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T07:57:10.954 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T07:57:10.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:10.954 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:10.955 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:10.955 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T07:57:10.955 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T07:57:10.955 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T07:57:10.955 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:11.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:11.135 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-23T07:57:11.184 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-23T07:57:11.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:11.184 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-03-23T07:57:11.234 INFO:teuthology.orchestra.run.smithi132.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-23T07:57:11.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:11.235 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:11.235 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:11.235 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-23T07:57:11.235 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-23T07:57:11.235 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:11.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:11.375 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-23T07:57:11.462 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:11.462 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether ca:5c:3d:63:ea:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::c85c:3dff:fe63:ea68/64 scope link 2024-03-23T07:57:11.463 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:11.467 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether ca:5c:3d:63:ea:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.558 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::c85c:3dff:fe63:ea68/64 scope link 2024-03-23T07:57:11.559 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:57:11.562 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-23T07:57:11.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-23T07:57:11.563 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:11.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:11.825 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-03-23T07:57:11.825 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-23T07:57:11.825 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.1 up 2024-03-23T07:57:11.825 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.1 master ceph-brx 2024-03-23T07:57:11.825 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-23T07:57:11.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:11.992 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-23T07:57:12.041 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-23T07:57:12.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:57:12.041 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-23T07:57:12.090 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-23T07:57:12.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:12.091 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:12.273 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:12.273 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:12.305 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:57:12.305 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:57:12.305 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-03-23T07:57:12.305 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with new device syntax: 3@e641b836-a0e5-4e95-93aa-bb0628868335.cephfs=/ 2024-03-23T07:57:12.305 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.105:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2024-03-23T07:57:12.308 INFO:teuthology.orchestra.run.smithi132.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-23T07:57:12.308 INFO:teuthology.orchestra.run.smithi132.stdout: You just mounted a file system that supports labels which does not 2024-03-23T07:57:12.308 INFO:teuthology.orchestra.run.smithi132.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T07:57:12.308 INFO:teuthology.orchestra.run.smithi132.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T07:57:12.308 INFO:teuthology.orchestra.run.smithi132.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T07:57:12.313 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T07:57:12.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:12.313 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-23T07:57:12.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:12.384 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:12.434 INFO:teuthology.orchestra.run.smithi132.stdout:client4642 2024-03-23T07:57:12.434 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:12.434 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:12.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:12.480 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:12.580 INFO:teuthology.orchestra.run.smithi132.stdout:client4642 2024-03-23T07:57:12.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:12.580 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:57:12.635 INFO:teuthology.orchestra.run.smithi132.stdout:client4642 2024-03-23T07:57:12.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:12.636 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e641b836-a0e5-4e95-93aa-bb0628868335.client4642/status') 2024-03-23T07:57:12.726 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.4642 (3)192.168.0.2:0/48218278 2024-03-23T07:57:12.727 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2024-03-23T07:57:12.727 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2024-03-23T07:57:12.727 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2024-03-23T07:57:12.727 INFO:teuthology.orchestra.run.smithi132.stderr:67 bytes copied, 8.3413e-05 s, 803 kB/s 2024-03-23T07:57:12.731 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-23T07:57:12.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.982+0000 7f4e66600640 1 Processor -- start 2024-03-23T07:57:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.982+0000 7f4e66600640 1 -- start start 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e66600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4e6015cab0 0x7f4e600b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e66600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4e600ceeb0 0x7f4e600cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e66600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 0x7f4e600b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e66600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e600cf850 con 0x7f4e600ceeb0 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e66600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6015ce90 con 0x7f4e6015cab0 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e66600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600cddf0 con 0x7f4e600b4120 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4e600ceeb0 0x7f4e600cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4e600ceeb0 0x7f4e600cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59866/0 (socket says 172.21.15.105:59866) 2024-03-23T07:57:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 -- 172.21.15.105:0/3343486804 learned_addr learned my addr 172.21.15.105:0/3343486804 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 -- 172.21.15.105:0/3343486804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 msgr2=0x7f4e600b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 --2- 172.21.15.105:0/3343486804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 0x7f4e600b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 -- 172.21.15.105:0/3343486804 >> v2:172.21.15.132:3300/0 conn(0x7f4e6015cab0 msgr2=0x7f4e600b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 --2- 172.21.15.105:0/3343486804 >> v2:172.21.15.132:3300/0 conn(0x7f4e6015cab0 0x7f4e600b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.984+0000 7f4e57e00640 1 -- 172.21.15.105:0/3343486804 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600cd570 con 0x7f4e600ceeb0 2024-03-23T07:57:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.985+0000 7f4e57e00640 1 --2- 172.21.15.105:0/3343486804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e600ceeb0 0x7f4e600cf310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e6008b100 tx=0x7f4e4802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23361e815f417bb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e5fe00640 1 -- 172.21.15.105:0/3343486804 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48031860 con 0x7f4e600ceeb0 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e5fe00640 1 -- 172.21.15.105:0/3343486804 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e480319c0 con 0x7f4e600ceeb0 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e5fe00640 1 -- 172.21.15.105:0/3343486804 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48031d50 con 0x7f4e600ceeb0 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e66600640 1 -- 172.21.15.105:0/3343486804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e600ceeb0 msgr2=0x7f4e600cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e66600640 1 --2- 172.21.15.105:0/3343486804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e600ceeb0 0x7f4e600cf310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e6008b100 tx=0x7f4e4802b900 comp rx=0 tx=0).stop 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e66600640 1 -- 172.21.15.105:0/3343486804 shutdown_connections 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e66600640 1 --2- 172.21.15.105:0/3343486804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 0x7f4e600b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e66600640 1 --2- 172.21.15.105:0/3343486804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e600ceeb0 0x7f4e600cf310 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e66600640 1 --2- 172.21.15.105:0/3343486804 >> v2:172.21.15.132:3300/0 conn(0x7f4e6015cab0 0x7f4e600b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.986+0000 7f4e66600640 1 -- 172.21.15.105:0/3343486804 >> 172.21.15.105:0/3343486804 conn(0x7f4e6006ae90 msgr2=0x7f4e6006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.987+0000 7f4e66600640 1 -- 172.21.15.105:0/3343486804 shutdown_connections 2024-03-23T07:57:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.987+0000 7f4e66600640 1 -- 172.21.15.105:0/3343486804 wait complete. 2024-03-23T07:57:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.988+0000 7f4e66600640 1 Processor -- start 2024-03-23T07:57:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.988+0000 7f4e66600640 1 -- start start 2024-03-23T07:57:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.989+0000 7f4e66600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 0x7f4e60147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.989+0000 7f4e66600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e600ceeb0 0x7f4e60156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.989+0000 7f4e66600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e6015cab0 0x7f4e60156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.989+0000 7f4e66600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e60148060 con 0x7f4e6015cab0 2024-03-23T07:57:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.989+0000 7f4e66600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6015d080 con 0x7f4e600ceeb0 2024-03-23T07:57:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.989+0000 7f4e66600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6015d360 con 0x7f4e600b4120 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e6015cab0 0x7f4e60156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e6015cab0 0x7f4e60156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59878/0 (socket says 172.21.15.105:59878) 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 -- 172.21.15.105:0/347360284 learned_addr learned my addr 172.21.15.105:0/347360284 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e57e00640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e600ceeb0 0x7f4e60156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 -- 172.21.15.105:0/347360284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 msgr2=0x7f4e60147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 0x7f4e60147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 -- 172.21.15.105:0/347360284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e600ceeb0 msgr2=0x7f4e60156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e600ceeb0 0x7f4e60156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 -- 172.21.15.105:0/347360284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480094d0 con 0x7f4e6015cab0 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e65c00640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e6015cab0 0x7f4e60156e80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b970 tx=0x7f4e5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.990+0000 7f4e5d600640 1 -- 172.21.15.105:0/347360284 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800cb40 con 0x7f4e6015cab0 2024-03-23T07:57:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.991+0000 7f4e5d600640 1 -- 172.21.15.105:0/347360284 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5800ccc0 con 0x7f4e6015cab0 2024-03-23T07:57:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.991+0000 7f4e5d600640 1 -- 172.21.15.105:0/347360284 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58014680 con 0x7f4e6015cab0 2024-03-23T07:57:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.991+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6015d640 con 0x7f4e6015cab0 2024-03-23T07:57:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.991+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6015dc00 con 0x7f4e6015cab0 2024-03-23T07:57:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.992+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6015ddf0 con 0x7f4e6015cab0 2024-03-23T07:57:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.993+0000 7f4e5d600640 1 -- 172.21.15.105:0/347360284 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4e5801d3b0 con 0x7f4e6015cab0 2024-03-23T07:57:12.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.994+0000 7f4e5d600640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e3c06ac00 0x7f4e3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:12.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.994+0000 7f4e5d600640 1 -- 172.21.15.105:0/347360284 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4e580896c0 con 0x7f4e6015cab0 2024-03-23T07:57:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.994+0000 7f4e65200640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e3c06ac00 0x7f4e3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.995+0000 7f4e65200640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e3c06ac00 0x7f4e3c06d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009710 tx=0x7f4e50006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:12.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:12.999+0000 7f4e5d600640 1 -- 172.21.15.105:0/347360284 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4e580544b0 con 0x7f4e6015cab0 2024-03-23T07:57:13.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.194+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e6004e030 con 0x7f4e6015cab0 2024-03-23T07:57:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.197+0000 7f4e5d600640 1 -- 172.21.15.105:0/347360284 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f4e58057ad0 con 0x7f4e6015cab0 2024-03-23T07:57:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:13.197 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.199+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e3c06ac00 msgr2=0x7f4e3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.199+0000 7f4e66600640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e3c06ac00 0x7f4e3c06d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009710 tx=0x7f4e50006c90 comp rx=0 tx=0).stop 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.199+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e6015cab0 msgr2=0x7f4e60156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.199+0000 7f4e66600640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e6015cab0 0x7f4e60156e80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b970 tx=0x7f4e5800be30 comp rx=0 tx=0).stop 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.199+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 shutdown_connections 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.200+0000 7f4e66600640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e6015cab0 0x7f4e60156e80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.200+0000 7f4e66600640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e600ceeb0 0x7f4e60156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.200+0000 7f4e66600640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e3c06ac00 0x7f4e3c06d0c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.200+0000 7f4e66600640 1 --2- 172.21.15.105:0/347360284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e600b4120 0x7f4e60147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.200+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 >> 172.21.15.105:0/347360284 conn(0x7f4e6006ae90 msgr2=0x7f4e6006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.200+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 shutdown_connections 2024-03-23T07:57:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.200+0000 7f4e66600640 1 -- 172.21.15.105:0/347360284 wait complete. 2024-03-23T07:57:13.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:13.219 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:13.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.427+0000 7f5bc3200640 1 Processor -- start 2024-03-23T07:57:13.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.427+0000 7f5bc3200640 1 -- start start 2024-03-23T07:57:13.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.428+0000 7f5bc3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc0b28b0 0x7f5bbc15d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.428+0000 7f5bc3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5bbc06b2f0 0x7f5bbc15d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.428+0000 7f5bc3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc066280 con 0x7f5bbc06af10 2024-03-23T07:57:13.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.428+0000 7f5bc3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cb650 con 0x7f5bbc06b2f0 2024-03-23T07:57:13.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.428+0000 7f5bc3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc15cf50 con 0x7f5bbc0b28b0 2024-03-23T07:57:13.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.429+0000 7f5bc1200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5bbc06b2f0 0x7f5bbc15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.429+0000 7f5bbb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc0b28b0 0x7f5bbc15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.429+0000 7f5bc1200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5bbc06b2f0 0x7f5bbc15d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:33828/0 (socket says 172.21.15.105:33828) 2024-03-23T07:57:13.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.429+0000 7f5bc1200640 1 -- 172.21.15.105:0/2157800340 learned_addr learned my addr 172.21.15.105:0/2157800340 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:13.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.430+0000 7f5bbb400640 1 -- 172.21.15.105:0/2157800340 >> v2:172.21.15.132:3300/0 conn(0x7f5bbc06b2f0 msgr2=0x7f5bbc15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:13.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.430+0000 7f5bbb400640 1 --2- 172.21.15.105:0/2157800340 >> v2:172.21.15.132:3300/0 conn(0x7f5bbc06b2f0 0x7f5bbc15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.430+0000 7f5bbb400640 1 -- 172.21.15.105:0/2157800340 >> v1:172.21.15.132:6789/0 conn(0x7f5bbc06af10 legacy=0x7f5bbc06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.430+0000 7f5bbb400640 1 -- 172.21.15.105:0/2157800340 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0b2500 con 0x7f5bbc0b28b0 2024-03-23T07:57:13.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.431+0000 7f5bbb400640 1 --2- 172.21.15.105:0/2157800340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc0b28b0 0x7f5bbc15d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80097a0 tx=0x7f5ba802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4869fd952b9ebdc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:13.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.432+0000 7f5bbaa00640 1 -- 172.21.15.105:0/2157800340 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8031780 con 0x7f5bbc0b28b0 2024-03-23T07:57:13.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.432+0000 7f5bbaa00640 1 -- 172.21.15.105:0/2157800340 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba80318e0 con 0x7f5bbc0b28b0 2024-03-23T07:57:13.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.432+0000 7f5bbaa00640 1 -- 172.21.15.105:0/2157800340 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8031c70 con 0x7f5bbc0b28b0 2024-03-23T07:57:13.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.432+0000 7f5bc3200640 1 -- 172.21.15.105:0/2157800340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc0b28b0 msgr2=0x7f5bbc15d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.432+0000 7f5bc3200640 1 --2- 172.21.15.105:0/2157800340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc0b28b0 0x7f5bbc15d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80097a0 tx=0x7f5ba802b820 comp rx=0 tx=0).stop 2024-03-23T07:57:13.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.433+0000 7f5bc3200640 1 -- 172.21.15.105:0/2157800340 shutdown_connections 2024-03-23T07:57:13.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.433+0000 7f5bc3200640 1 --2- 172.21.15.105:0/2157800340 >> v2:172.21.15.132:3300/0 conn(0x7f5bbc06b2f0 0x7f5bbc15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.433+0000 7f5bc3200640 1 --2- 172.21.15.105:0/2157800340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc0b28b0 0x7f5bbc15d420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.433+0000 7f5bc3200640 1 -- 172.21.15.105:0/2157800340 >> 172.21.15.105:0/2157800340 conn(0x7f5bbc0709c0 msgr2=0x7f5bbc070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:13.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.433+0000 7f5bc3200640 1 -- 172.21.15.105:0/2157800340 shutdown_connections 2024-03-23T07:57:13.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.433+0000 7f5bc3200640 1 -- 172.21.15.105:0/2157800340 wait complete. 2024-03-23T07:57:13.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.435+0000 7f5bc3200640 1 Processor -- start 2024-03-23T07:57:13.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.435+0000 7f5bc3200640 1 -- start start 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bbc06af10 0x7f5bbc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc06b2f0 0x7f5bbc0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc0b28b0 0x7f5bbc1427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0ce730 con 0x7f5bbc06af10 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc142d10 con 0x7f5bbc0b28b0 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc15d9e0 con 0x7f5bbc06b2f0 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bbc06af10 0x7f5bbc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bbc06af10 0x7f5bbc15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59888/0 (socket says 172.21.15.105:59888) 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 -- 172.21.15.105:0/1346561594 learned_addr learned my addr 172.21.15.105:0/1346561594 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbb400640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc06b2f0 0x7f5bbc0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc1200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc0b28b0 0x7f5bbc1427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 -- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc06b2f0 msgr2=0x7f5bbc0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc06b2f0 0x7f5bbc0ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 -- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc0b28b0 msgr2=0x7f5bbc1427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc0b28b0 0x7f5bbc1427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbbe00640 1 -- 172.21.15.105:0/1346561594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8009400 con 0x7f5bbc06af10 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bc1200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc0b28b0 0x7f5bbc1427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.436+0000 7f5bbb400640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc06b2f0 0x7f5bbc0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.437+0000 7f5bbbe00640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bbc06af10 0x7f5bbc15d320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00c3c0 tx=0x7f5bac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.437+0000 7f5bb3e00640 1 -- 172.21.15.105:0/1346561594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0052d0 con 0x7f5bbc06af10 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.437+0000 7f5bb3e00640 1 -- 172.21.15.105:0/1346561594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac005450 con 0x7f5bbc06af10 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.437+0000 7f5bb3e00640 1 -- 172.21.15.105:0/1346561594 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac007260 con 0x7f5bbc06af10 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.437+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc15dcc0 con 0x7f5bbc06af10 2024-03-23T07:57:13.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.437+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc15e280 con 0x7f5bbc06af10 2024-03-23T07:57:13.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.438+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc073390 con 0x7f5bbc06af10 2024-03-23T07:57:13.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.439+0000 7f5bb3e00640 1 -- 172.21.15.105:0/1346561594 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5bac0073c0 con 0x7f5bbc06af10 2024-03-23T07:57:13.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.440+0000 7f5bb3e00640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5b9c06ac00 0x7f5b9c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.440+0000 7f5bb3e00640 1 -- 172.21.15.105:0/1346561594 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5bac0888b0 con 0x7f5bbc06af10 2024-03-23T07:57:13.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.440+0000 7f5bbb400640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5b9c06ac00 0x7f5b9c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.441+0000 7f5bbb400640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5b9c06ac00 0x7f5b9c06d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8036000 tx=0x7f5ba803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:13.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.445+0000 7f5bb3e00640 1 -- 172.21.15.105:0/1346561594 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5bac053750 con 0x7f5bbc06af10 2024-03-23T07:57:13.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.639+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bbc066e60 con 0x7f5bbc06af10 2024-03-23T07:57:13.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.640+0000 7f5bb3e00640 1 -- 172.21.15.105:0/1346561594 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bac056d70 con 0x7f5bbc06af10 2024-03-23T07:57:13.640 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.642+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5b9c06ac00 msgr2=0x7f5b9c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.642+0000 7f5bc3200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5b9c06ac00 0x7f5b9c06d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8036000 tx=0x7f5ba803d040 comp rx=0 tx=0).stop 2024-03-23T07:57:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.642+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bbc06af10 msgr2=0x7f5bbc15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.642+0000 7f5bc3200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bbc06af10 0x7f5bbc15d320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00c3c0 tx=0x7f5bac00c880 comp rx=0 tx=0).stop 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 shutdown_connections 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc0b28b0 0x7f5bbc1427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5b9c06ac00 0x7f5b9c06d0c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bbc06b2f0 0x7f5bbc0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 --2- 172.21.15.105:0/1346561594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bbc06af10 0x7f5bbc15d320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 >> 172.21.15.105:0/1346561594 conn(0x7f5bbc0709c0 msgr2=0x7f5bbc06d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 shutdown_connections 2024-03-23T07:57:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.643+0000 7f5bc3200640 1 -- 172.21.15.105:0/1346561594 wait complete. 2024-03-23T07:57:13.661 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:13.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.877+0000 7fd2b0200640 1 Processor -- start 2024-03-23T07:57:13.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.877+0000 7fd2b0200640 1 -- start start 2024-03-23T07:57:13.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.878+0000 7fd2b0200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2a8098a40 0x7fd2a8098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2b0200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 0x7fd2a8165940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2b0200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a8099360 con 0x7fd2a80cf120 2024-03-23T07:57:13.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2b0200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a80cb5d0 con 0x7fd2a8098a40 2024-03-23T07:57:13.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2b0200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80cb8b0 con 0x7fd2a815ed20 2024-03-23T07:57:13.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2ace00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd2a80cf120 0x7fd2a815e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40470/0 (socket says 172.21.15.105:40470) 2024-03-23T07:57:13.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2ace00640 1 -- 172.21.15.105:0/1621410939 learned_addr learned my addr 172.21.15.105:0/1621410939 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2ad800640 1 --2- 172.21.15.105:0/1621410939 >> v2:172.21.15.132:3300/0 conn(0x7fd2a8098a40 0x7fd2a8098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2ae200640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 0x7fd2a8165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836161223 0 0) 0x7fd2a8099360 con 0x7fd2a80cf120 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.879+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd28c003520 con 0x7fd2a80cf120 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 350789281 0 0) 0x7fd28c003520 con 0x7fd2a80cf120 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 msgr2=0x7fd2a8165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 0x7fd2a8165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 >> v2:172.21.15.132:3300/0 conn(0x7fd2a8098a40 msgr2=0x7fd2a8098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 --2- 172.21.15.105:0/1621410939 >> v2:172.21.15.132:3300/0 conn(0x7fd2a8098a40 0x7fd2a8098e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80cbb90 con 0x7fd2a80cf120 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fd294001670 con 0x7fd2a80cf120 2024-03-23T07:57:13.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 >> v1:172.21.15.105:6789/0 conn(0x7fd2a80cf120 legacy=0x7fd2a815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 0x7fd28c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd28c003f40 0x7fd28c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c004d00 0x7fd28c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80cb5d0 con 0x7fd28c003f40 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80cb8b0 con 0x7fd28c004d00 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00b980 con 0x7fd2a815ed20 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.880+0000 7fd2ace00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd28c003f40 0x7fd28c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ad800640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 0x7fd28c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ae200640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c004d00 0x7fd28c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ace00640 1 -- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 msgr2=0x7fd28c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ace00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a815ed20 0x7fd28c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ace00640 1 -- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c004d00 msgr2=0x7fd28c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ace00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c004d00 0x7fd28c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ace00640 1 -- 172.21.15.105:0/1621410939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c003520 con 0x7fd28c003f40 2024-03-23T07:57:13.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.881+0000 7fd2ae200640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c004d00 0x7fd28c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:13.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.882+0000 7fd2ace00640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd28c003f40 0x7fd28c0047c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c190 tx=0x7fd28c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910a31c8246e1c21 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.882+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294005ad0 con 0x7fd28c003f40 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.882+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd294005c30 con 0x7fd28c003f40 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.882+0000 7fd2a7e00640 1 -- 172.21.15.105:0/1621410939 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29402c810 con 0x7fd28c003f40 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.883+0000 7fd2b0200640 1 -- 172.21.15.105:0/1621410939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd28c003f40 msgr2=0x7fd28c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.883+0000 7fd2b0200640 1 --2- 172.21.15.105:0/1621410939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd28c003f40 0x7fd28c0047c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c190 tx=0x7fd28c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.883+0000 7fd2ae200640 1 -- 172.21.15.105:0/1621410939 reap_dead start 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.883+0000 7fd2b0200640 1 -- 172.21.15.105:0/1621410939 shutdown_connections 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.883+0000 7fd2b0200640 1 -- 172.21.15.105:0/1621410939 >> 172.21.15.105:0/1621410939 conn(0x7fd2a80b2180 msgr2=0x7fd2a80b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.883+0000 7fd2b0200640 1 -- 172.21.15.105:0/1621410939 shutdown_connections 2024-03-23T07:57:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.883+0000 7fd2b0200640 1 -- 172.21.15.105:0/1621410939 wait complete. 2024-03-23T07:57:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.885+0000 7fd2b0200640 1 Processor -- start 2024-03-23T07:57:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.885+0000 7fd2b0200640 1 -- start start 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2b0200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c003f40 0x7fd2a80c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2b0200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a80ca0e0 0x7fd2a8159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2b0200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2a80c8ae0 0x7fd2a80c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2b0200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8157470 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2b0200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80c7760 con 0x7fd28c003f40 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2b0200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80c7a40 con 0x7fd2a80ca0e0 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2ae200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2a80c8ae0 0x7fd2a80c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2ae200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2a80c8ae0 0x7fd2a80c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59910/0 (socket says 172.21.15.105:59910) 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2ae200640 1 -- 172.21.15.105:0/1480459480 learned_addr learned my addr 172.21.15.105:0/1480459480 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.886+0000 7fd2ad800640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c003f40 0x7fd2a80c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2ace00640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a80ca0e0 0x7fd2a8159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2ae200640 1 -- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a80ca0e0 msgr2=0x7fd2a8159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2ae200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a80ca0e0 0x7fd2a8159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2ae200640 1 -- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c003f40 msgr2=0x7fd2a80c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2ae200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c003f40 0x7fd2a80c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2ae200640 1 -- 172.21.15.105:0/1480459480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80c7d20 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2ae200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2a80c8ae0 0x7fd2a80c8f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8072350 tx=0x7fd2a0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2a5600640 1 -- 172.21.15.105:0/1480459480 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000e8a0 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2a5600640 1 -- 172.21.15.105:0/1480459480 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000ea20 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2a5600640 1 -- 172.21.15.105:0/1480459480 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000edd0 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a80c8000 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.887+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a80c85c0 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.889+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80b1f80 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.890+0000 7fd2a5600640 1 -- 172.21.15.105:0/1480459480 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd2a0020410 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.891+0000 7fd2a5600640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd28806ac00 0x7fd28806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.891+0000 7fd2a5600640 1 -- 172.21.15.105:0/1480459480 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd2a0088fe0 con 0x7fd2a80c8ae0 2024-03-23T07:57:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.891+0000 7fd2ad800640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd28806ac00 0x7fd28806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.892+0000 7fd2ad800640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd28806ac00 0x7fd28806d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd298007520 tx=0x7fd29800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:13.895+0000 7fd2a5600640 1 -- 172.21.15.105:0/1480459480 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd2a0053e30 con 0x7fd2a80c8ae0 2024-03-23T07:57:14.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.088+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2a80cfc70 con 0x7fd2a80c8ae0 2024-03-23T07:57:14.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.089+0000 7fd2a5600640 1 -- 172.21.15.105:0/1480459480 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2a0006c30 con 0x7fd2a80c8ae0 2024-03-23T07:57:14.089 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd28806ac00 msgr2=0x7fd28806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd28806ac00 0x7fd28806d0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd298007520 tx=0x7fd29800b040 comp rx=0 tx=0).stop 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2a80c8ae0 msgr2=0x7fd2a80c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2a80c8ae0 0x7fd2a80c8f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8072350 tx=0x7fd2a0010c20 comp rx=0 tx=0).stop 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 shutdown_connections 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2a80c8ae0 0x7fd2a80c8f60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd28806ac00 0x7fd28806d0c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a80ca0e0 0x7fd2a8159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 --2- 172.21.15.105:0/1480459480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd28c003f40 0x7fd2a80c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 >> 172.21.15.105:0/1480459480 conn(0x7fd2a80b2180 msgr2=0x7fd2a8158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 shutdown_connections 2024-03-23T07:57:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.092+0000 7fd2b0200640 1 -- 172.21.15.105:0/1480459480 wait complete. 2024-03-23T07:57:14.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:14.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.316+0000 7f3ded200640 1 Processor -- start 2024-03-23T07:57:14.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.316+0000 7f3ded200640 1 -- start start 2024-03-23T07:57:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ded200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ded200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3de8070cd0 0x7f3de80cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ded200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de806d390 con 0x7f3de8145c20 2024-03-23T07:57:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ded200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de80ad890 con 0x7f3de8070cd0 2024-03-23T07:57:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ded200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8145a80 con 0x7f3de81456a0 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ddd400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3de8070cd0 0x7f3de80cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ddd400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3de8070cd0 0x7f3de80cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59918/0 (socket says 172.21.15.105:59918) 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.318+0000 7f3ddd400640 1 -- 172.21.15.105:0/1937072779 learned_addr learned my addr 172.21.15.105:0/1937072779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.319+0000 7f3de5e00640 1 --2- 172.21.15.105:0/1937072779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.319+0000 7f3ddd400640 1 -- 172.21.15.105:0/1937072779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 msgr2=0x7f3de80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.319+0000 7f3ddd400640 1 --2- 172.21.15.105:0/1937072779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.319+0000 7f3ddd400640 1 -- 172.21.15.105:0/1937072779 >> v1:172.21.15.105:6789/0 conn(0x7f3de8145c20 legacy=0x7f3de80ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.319+0000 7f3de5400640 1 -- 172.21.15.105:0/1937072779 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560209693 0 0) 0x7f3de806d390 con 0x7f3de8145c20 2024-03-23T07:57:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.319+0000 7f3ddd400640 1 -- 172.21.15.105:0/1937072779 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de806d670 con 0x7f3de8070cd0 2024-03-23T07:57:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.320+0000 7f3ddd400640 1 --2- 172.21.15.105:0/1937072779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de80cee70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940a0914ed8ec277 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3de5400640 1 -- 172.21.15.105:0/1937072779 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0030840 con 0x7f3de8070cd0 2024-03-23T07:57:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3de5400640 1 -- 172.21.15.105:0/1937072779 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd00309a0 con 0x7f3de8070cd0 2024-03-23T07:57:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3de5400640 1 -- 172.21.15.105:0/1937072779 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0030d30 con 0x7f3de8070cd0 2024-03-23T07:57:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 -- 172.21.15.105:0/1937072779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 msgr2=0x7f3de80cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 --2- 172.21.15.105:0/1937072779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de80cee70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd002b980 comp rx=0 tx=0).stop 2024-03-23T07:57:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 -- 172.21.15.105:0/1937072779 shutdown_connections 2024-03-23T07:57:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 --2- 172.21.15.105:0/1937072779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de80cee70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 --2- 172.21.15.105:0/1937072779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 -- 172.21.15.105:0/1937072779 >> 172.21.15.105:0/1937072779 conn(0x7f3de806a0b0 msgr2=0x7f3de806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 -- 172.21.15.105:0/1937072779 shutdown_connections 2024-03-23T07:57:14.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.321+0000 7f3ded200640 1 -- 172.21.15.105:0/1937072779 wait complete. 2024-03-23T07:57:14.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.323+0000 7f3ded200640 1 Processor -- start 2024-03-23T07:57:14.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.323+0000 7f3ded200640 1 -- start start 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.324+0000 7f3ded200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.324+0000 7f3ded200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.324+0000 7f3ded200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de8145c20 0x7f3de815d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.324+0000 7f3ded200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80ae2c0 con 0x7f3de8070cd0 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.324+0000 7f3ded200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815c920 con 0x7f3de8145c20 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.324+0000 7f3ded200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80aedd0 con 0x7f3de81456a0 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de815c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59924/0 (socket says 172.21.15.105:59924) 2024-03-23T07:57:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 -- 172.21.15.105:0/2056094403 learned_addr learned my addr 172.21.15.105:0/2056094403 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de6800640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de8145c20 0x7f3de815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3ddd400640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 -- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 msgr2=0x7f3de815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de815ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 -- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de8145c20 msgr2=0x7f3de815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de8145c20 0x7f3de815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 -- 172.21.15.105:0/2056094403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00094d0 con 0x7f3de8070cd0 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3ddd400640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de815ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de6800640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de8145c20 0x7f3de815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.325+0000 7f3de5e00640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de815c3e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800d830 tx=0x7f3dd800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.326+0000 7f3ddea00640 1 -- 172.21.15.105:0/2056094403 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800b6c0 con 0x7f3de8070cd0 2024-03-23T07:57:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.326+0000 7f3ddea00640 1 -- 172.21.15.105:0/2056094403 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800b820 con 0x7f3de8070cd0 2024-03-23T07:57:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.326+0000 7f3ddea00640 1 -- 172.21.15.105:0/2056094403 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800bbb0 con 0x7f3de8070cd0 2024-03-23T07:57:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.326+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de80af260 con 0x7f3de8070cd0 2024-03-23T07:57:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.326+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de80ae780 con 0x7f3de8070cd0 2024-03-23T07:57:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.327+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de8142700 con 0x7f3de8070cd0 2024-03-23T07:57:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.328+0000 7f3ddea00640 1 -- 172.21.15.105:0/2056094403 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3dd801e460 con 0x7f3de8070cd0 2024-03-23T07:57:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.329+0000 7f3ddea00640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3dc406ac30 0x7f3dc406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.329+0000 7f3ddea00640 1 -- 172.21.15.105:0/2056094403 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3dd8089ec0 con 0x7f3de8070cd0 2024-03-23T07:57:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.329+0000 7f3ddd400640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3dc406ac30 0x7f3dc406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.330+0000 7f3ddd400640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3dc406ac30 0x7f3dc406d0f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0036000 tx=0x7f3dd003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:14.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.333+0000 7f3ddea00640 1 -- 172.21.15.105:0/2056094403 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3dd8054cb0 con 0x7f3de8070cd0 2024-03-23T07:57:14.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.527+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3de8073390 con 0x7f3de8070cd0 2024-03-23T07:57:14.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.528+0000 7f3ddea00640 1 -- 172.21.15.105:0/2056094403 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3dd80582d0 con 0x7f3de8070cd0 2024-03-23T07:57:14.527 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.530+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3dc406ac30 msgr2=0x7f3dc406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.530+0000 7f3ded200640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3dc406ac30 0x7f3dc406d0f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0036000 tx=0x7f3dd003b040 comp rx=0 tx=0).stop 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.530+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 msgr2=0x7f3de815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.530+0000 7f3ded200640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de815c3e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800d830 tx=0x7f3dd800dcf0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 shutdown_connections 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de8145c20 0x7f3de815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3dc406ac30 0x7f3dc406d0f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de81456a0 0x7f3de815ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 --2- 172.21.15.105:0/2056094403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8070cd0 0x7f3de815c3e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 >> 172.21.15.105:0/2056094403 conn(0x7f3de806a0b0 msgr2=0x7f3de8140070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 shutdown_connections 2024-03-23T07:57:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.531+0000 7f3ded200640 1 -- 172.21.15.105:0/2056094403 wait complete. 2024-03-23T07:57:14.550 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:14.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.755+0000 7f80e7e00640 1 Processor -- start 2024-03-23T07:57:14.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.755+0000 7f80e7e00640 1 -- start start 2024-03-23T07:57:14.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e7e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80e00cd740 0x7f80e00b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e00d0d40 0x7f80e00cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e7e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e00d20c0 con 0x7f80e006b0a0 2024-03-23T07:57:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e7e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e00ce060 con 0x7f80e00cd740 2024-03-23T07:57:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00d14b0 con 0x7f80e00d0d40 2024-03-23T07:57:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e5400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80e00cd740 0x7f80e00b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e5e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f80e006b0a0 0x7f80e006b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40492/0 (socket says 172.21.15.105:40492) 2024-03-23T07:57:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.757+0000 7f80e5e00640 1 -- 172.21.15.105:0/2593408039 learned_addr learned my addr 172.21.15.105:0/2593408039 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.758+0000 7f80e4a00640 1 --2- 172.21.15.105:0/2593408039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e00d0d40 0x7f80e00cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.758+0000 7f80e5400640 1 -- 172.21.15.105:0/2593408039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e00d0d40 msgr2=0x7f80e00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.758+0000 7f80e5400640 1 --2- 172.21.15.105:0/2593408039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e00d0d40 0x7f80e00cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.758+0000 7f80e5400640 1 -- 172.21.15.105:0/2593408039 >> v1:172.21.15.105:6789/0 conn(0x7f80e006b0a0 legacy=0x7f80e006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.758+0000 7f80e5400640 1 -- 172.21.15.105:0/2593408039 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00d1790 con 0x7f80e00cd740 2024-03-23T07:57:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.759+0000 7f80e5400640 1 --2- 172.21.15.105:0/2593408039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00cd740 0x7f80e00b84b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f80d00097a0 tx=0x7f80d002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52de53f5b9689e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.759+0000 7f80dfe00640 1 -- 172.21.15.105:0/2593408039 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d002d040 con 0x7f80e00cd740 2024-03-23T07:57:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.759+0000 7f80dfe00640 1 -- 172.21.15.105:0/2593408039 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0031810 con 0x7f80e00cd740 2024-03-23T07:57:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.759+0000 7f80dfe00640 1 -- 172.21.15.105:0/2593408039 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0031ba0 con 0x7f80e00cd740 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 -- 172.21.15.105:0/2593408039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00cd740 msgr2=0x7f80e00b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 --2- 172.21.15.105:0/2593408039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00cd740 0x7f80e00b84b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f80d00097a0 tx=0x7f80d002b8b0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 -- 172.21.15.105:0/2593408039 shutdown_connections 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 --2- 172.21.15.105:0/2593408039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e00d0d40 0x7f80e00cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 --2- 172.21.15.105:0/2593408039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00cd740 0x7f80e00b84b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 -- 172.21.15.105:0/2593408039 >> 172.21.15.105:0/2593408039 conn(0x7f80e00cf5d0 msgr2=0x7f80e00cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 -- 172.21.15.105:0/2593408039 shutdown_connections 2024-03-23T07:57:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.760+0000 7f80e7e00640 1 -- 172.21.15.105:0/2593408039 wait complete. 2024-03-23T07:57:14.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.762+0000 7f80e7e00640 1 Processor -- start 2024-03-23T07:57:14.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.762+0000 7f80e7e00640 1 -- start start 2024-03-23T07:57:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e006b0a0 0x7f80e015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80e00cd740 0x7f80e00ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00d0d40 0x7f80e00cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e7e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00ab7f0 con 0x7f80e00d0d40 2024-03-23T07:57:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00aa210 con 0x7f80e00cd740 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00aa4f0 con 0x7f80e006b0a0 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00d0d40 0x7f80e00cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00d0d40 0x7f80e00cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59942/0 (socket says 172.21.15.105:59942) 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 -- 172.21.15.105:0/3345144867 learned_addr learned my addr 172.21.15.105:0/3345144867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 -- 172.21.15.105:0/3345144867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e006b0a0 msgr2=0x7f80e015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e006b0a0 0x7f80e015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 -- 172.21.15.105:0/3345144867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80e00cd740 msgr2=0x7f80e00ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80e00cd740 0x7f80e00ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.763+0000 7f80e5e00640 1 -- 172.21.15.105:0/3345144867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d0009400 con 0x7f80e00d0d40 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.764+0000 7f80e5e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00d0d40 0x7f80e00cba50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c420 tx=0x7f80d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.764+0000 7f80dd600640 1 -- 172.21.15.105:0/3345144867 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8005ce0 con 0x7f80e00d0d40 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.764+0000 7f80dd600640 1 -- 172.21.15.105:0/3345144867 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8005e40 con 0x7f80e00d0d40 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.764+0000 7f80dd600640 1 -- 172.21.15.105:0/3345144867 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80072d0 con 0x7f80e00d0d40 2024-03-23T07:57:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.764+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e00aa7d0 con 0x7f80e00d0d40 2024-03-23T07:57:14.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.764+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e00aad90 con 0x7f80e00d0d40 2024-03-23T07:57:14.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.765+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e00aafa0 con 0x7f80e00d0d40 2024-03-23T07:57:14.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.766+0000 7f80dd600640 1 -- 172.21.15.105:0/3345144867 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f80d80074f0 con 0x7f80e00d0d40 2024-03-23T07:57:14.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.767+0000 7f80dd600640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80c006ac00 0x7f80c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:14.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.767+0000 7f80dd600640 1 -- 172.21.15.105:0/3345144867 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f80d8088b10 con 0x7f80e00d0d40 2024-03-23T07:57:14.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.767+0000 7f80e5400640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80c006ac00 0x7f80c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:14.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.768+0000 7f80e5400640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80c006ac00 0x7f80c006d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f80d0036000 tx=0x7f80d0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:14.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.771+0000 7f80dd600640 1 -- 172.21.15.105:0/3345144867 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f80d8053ae0 con 0x7f80e00d0d40 2024-03-23T07:57:14.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.967+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f80e0073390 con 0x7f80e00d0d40 2024-03-23T07:57:14.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.968+0000 7f80dd600640 1 -- 172.21.15.105:0/3345144867 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f80d8017020 con 0x7f80e00d0d40 2024-03-23T07:57:14.967 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.970+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80c006ac00 msgr2=0x7f80c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.970+0000 7f80e7e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80c006ac00 0x7f80c006d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f80d0036000 tx=0x7f80d0030000 comp rx=0 tx=0).stop 2024-03-23T07:57:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.970+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00d0d40 msgr2=0x7f80e00cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.970+0000 7f80e7e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00d0d40 0x7f80e00cba50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c420 tx=0x7f80d800c8e0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 shutdown_connections 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00d0d40 0x7f80e00cba50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80e00cd740 0x7f80e00ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80c006ac00 0x7f80c006d0c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 --2- 172.21.15.105:0/3345144867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80e006b0a0 0x7f80e015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 >> 172.21.15.105:0/3345144867 conn(0x7f80e00cf5d0 msgr2=0x7f80e00b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 shutdown_connections 2024-03-23T07:57:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:14.971+0000 7f80e7e00640 1 -- 172.21.15.105:0/3345144867 wait complete. 2024-03-23T07:57:14.989 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:15.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.198+0000 7f2691400640 1 Processor -- start 2024-03-23T07:57:15.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.199+0000 7f2691400640 1 -- start start 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f2691400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f268c15cd70 0x7f268c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f2691400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f268c0b4a50 0x7f268c0b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f2691400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0d17d0 con 0x7f268c0d1330 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f2691400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c15d150 con 0x7f268c0b4a50 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f2691400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0d1ab0 con 0x7f268c15cd70 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f268b200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f268c0b4a50 0x7f268c0b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f268b200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f268c0b4a50 0x7f268c0b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59956/0 (socket says 172.21.15.105:59956) 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f268b200640 1 -- 172.21.15.105:0/2729086671 learned_addr learned my addr 172.21.15.105:0/2729086671 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:15.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.200+0000 7f268a800640 1 --2- 172.21.15.105:0/2729086671 >> v2:172.21.15.132:3300/0 conn(0x7f268c15cd70 0x7f268c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.201+0000 7f268b200640 1 -- 172.21.15.105:0/2729086671 >> v2:172.21.15.132:3300/0 conn(0x7f268c15cd70 msgr2=0x7f268c0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:15.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.201+0000 7f268b200640 1 --2- 172.21.15.105:0/2729086671 >> v2:172.21.15.132:3300/0 conn(0x7f268c15cd70 0x7f268c0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.201+0000 7f268b200640 1 -- 172.21.15.105:0/2729086671 >> v1:172.21.15.132:6789/0 conn(0x7f268c0d1330 legacy=0x7f268c0b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:15.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.201+0000 7f268b200640 1 -- 172.21.15.105:0/2729086671 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0d1d90 con 0x7f268c0b4a50 2024-03-23T07:57:15.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.202+0000 7f268b200640 1 --2- 172.21.15.105:0/2729086671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c0b4a50 0x7f268c0b5ba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f268c08b520 tx=0x7f268402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ef012db2b40674 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:15.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.202+0000 7f2689400640 1 -- 172.21.15.105:0/2729086671 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840317e0 con 0x7f268c0b4a50 2024-03-23T07:57:15.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.202+0000 7f2689400640 1 -- 172.21.15.105:0/2729086671 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684031940 con 0x7f268c0b4a50 2024-03-23T07:57:15.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.202+0000 7f2689400640 1 -- 172.21.15.105:0/2729086671 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684031cd0 con 0x7f268c0b4a50 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 -- 172.21.15.105:0/2729086671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c0b4a50 msgr2=0x7f268c0b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 --2- 172.21.15.105:0/2729086671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c0b4a50 0x7f268c0b5ba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f268c08b520 tx=0x7f268402b880 comp rx=0 tx=0).stop 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 -- 172.21.15.105:0/2729086671 shutdown_connections 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 --2- 172.21.15.105:0/2729086671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c0b4a50 0x7f268c0b5ba0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 --2- 172.21.15.105:0/2729086671 >> v2:172.21.15.132:3300/0 conn(0x7f268c15cd70 0x7f268c0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 -- 172.21.15.105:0/2729086671 >> 172.21.15.105:0/2729086671 conn(0x7f268c0cf340 msgr2=0x7f268c0cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 -- 172.21.15.105:0/2729086671 shutdown_connections 2024-03-23T07:57:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.203+0000 7f2691400640 1 -- 172.21.15.105:0/2729086671 wait complete. 2024-03-23T07:57:15.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.204+0000 7f2691400640 1 Processor -- start 2024-03-23T07:57:15.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.205+0000 7f2691400640 1 -- start start 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.205+0000 7f2691400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0b4a50 0x7f268c0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f2691400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c0d1330 0x7f268c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f2691400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c15cd70 0x7f268c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f2691400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c14f390 con 0x7f268c15cd70 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f2691400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0cb0a0 con 0x7f268c0b4a50 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f2691400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0cb380 con 0x7f268c0d1330 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c15cd70 0x7f268c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c15cd70 0x7f268c0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59970/0 (socket says 172.21.15.105:59970) 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 -- 172.21.15.105:0/2953938915 learned_addr learned my addr 172.21.15.105:0/2953938915 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f2689e00640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c0d1330 0x7f268c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268a800640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0b4a50 0x7f268c0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 -- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c0d1330 msgr2=0x7f268c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c0d1330 0x7f268c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 -- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0b4a50 msgr2=0x7f268c0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0b4a50 0x7f268c0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268b200640 1 -- 172.21.15.105:0/2953938915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840094d0 con 0x7f268c15cd70 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f2689e00640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c0d1330 0x7f268c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.206+0000 7f268a800640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0b4a50 0x7f268c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:15.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.207+0000 7f268b200640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c15cd70 0x7f268c0abf90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f26840090b0 tx=0x7f26840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:15.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.207+0000 7f2682a00640 1 -- 172.21.15.105:0/2953938915 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684032880 con 0x7f268c15cd70 2024-03-23T07:57:15.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.207+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c0cb660 con 0x7f268c15cd70 2024-03-23T07:57:15.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.207+0000 7f2682a00640 1 -- 172.21.15.105:0/2953938915 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684032a00 con 0x7f268c15cd70 2024-03-23T07:57:15.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.207+0000 7f2682a00640 1 -- 172.21.15.105:0/2953938915 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684032db0 con 0x7f268c15cd70 2024-03-23T07:57:15.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.207+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c0cbc20 con 0x7f268c15cd70 2024-03-23T07:57:15.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.208+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c0cbe10 con 0x7f268c15cd70 2024-03-23T07:57:15.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.209+0000 7f2682a00640 1 -- 172.21.15.105:0/2953938915 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f268404d3f0 con 0x7f268c15cd70 2024-03-23T07:57:15.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.210+0000 7f2682a00640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f266806ac00 0x7f266806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.210+0000 7f2682a00640 1 -- 172.21.15.105:0/2953938915 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f26840ab4d0 con 0x7f268c15cd70 2024-03-23T07:57:15.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.210+0000 7f268a800640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f266806ac00 0x7f266806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.211+0000 7f268a800640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f266806ac00 0x7f266806d0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2674006fd0 tx=0x7f2674008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:15.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.214+0000 7f2682a00640 1 -- 172.21.15.105:0/2953938915 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2684076240 con 0x7f268c15cd70 2024-03-23T07:57:15.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.408+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f268c073390 con 0x7f268c15cd70 2024-03-23T07:57:15.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.410+0000 7f2682a00640 1 -- 172.21.15.105:0/2953938915 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f2684079860 con 0x7f268c15cd70 2024-03-23T07:57:15.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:15.411 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:15.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f266806ac00 msgr2=0x7f266806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f266806ac00 0x7f266806d0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2674006fd0 tx=0x7f2674008040 comp rx=0 tx=0).stop 2024-03-23T07:57:15.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c15cd70 msgr2=0x7f268c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c15cd70 0x7f268c0abf90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f26840090b0 tx=0x7f26840046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 shutdown_connections 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f268c15cd70 0x7f268c0abf90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f266806ac00 0x7f266806d0c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c0d1330 0x7f268c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 --2- 172.21.15.105:0/2953938915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0b4a50 0x7f268c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.413+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 >> 172.21.15.105:0/2953938915 conn(0x7f268c0cf340 msgr2=0x7f268c0b7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.414+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 shutdown_connections 2024-03-23T07:57:15.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.414+0000 7f2691400640 1 -- 172.21.15.105:0/2953938915 wait complete. 2024-03-23T07:57:15.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:15.432 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:15.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.643+0000 7f730e800640 1 Processor -- start 2024-03-23T07:57:15.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.644+0000 7f730e800640 1 -- start start 2024-03-23T07:57:15.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.645+0000 7f730e800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f73080d0d40 0x7f73080b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.645+0000 7f730e800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080d20c0 con 0x7f73080cd740 2024-03-23T07:57:15.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.645+0000 7f730e800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080b8940 con 0x7f73080b84b0 2024-03-23T07:57:15.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.645+0000 7f730e800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73080d14b0 con 0x7f73080d0d40 2024-03-23T07:57:15.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.645+0000 7f7307400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f73080cd740 0x7f73080cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40496/0 (socket says 172.21.15.105:40496) 2024-03-23T07:57:15.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.645+0000 7f7307400640 1 -- 172.21.15.105:0/4101313795 learned_addr learned my addr 172.21.15.105:0/4101313795 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918816322 0 0) 0x7f73080d20c0 con 0x7f73080cd740 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7307e00640 1 --2- 172.21.15.105:0/4101313795 >> v2:172.21.15.132:3300/0 conn(0x7f73080d0d40 0x7f73080b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72ec003520 con 0x7f73080cd740 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 597153952 0 0) 0x7f72ec003520 con 0x7f73080cd740 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 >> v2:172.21.15.132:3300/0 conn(0x7f73080d0d40 msgr2=0x7f73080b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 --2- 172.21.15.105:0/4101313795 >> v2:172.21.15.132:3300/0 conn(0x7f73080d0d40 0x7f73080b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 >> v1:172.21.15.132:6789/0 conn(0x7f73080b84b0 legacy=0x7f73080b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080d1790 con 0x7f73080cd740 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f72f8001670 con 0x7f73080cd740 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 >> v1:172.21.15.105:6789/0 conn(0x7f73080cd740 legacy=0x7f73080cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f72ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ec0047c0 0x7f72ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72ec0051d0 0x7f72ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080d14b0 con 0x7f72ec0039c0 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec00c100 con 0x7f72ec0047c0 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec00c2c0 con 0x7f72ec0051d0 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.646+0000 7f7307400640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f72ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f7307400640 1 -- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72ec0051d0 msgr2=0x7f72ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f72fea00640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ec0047c0 0x7f72ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f7307400640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72ec0051d0 0x7f72ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f7307400640 1 -- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ec0047c0 msgr2=0x7f72ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f7307400640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ec0047c0 0x7f72ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f7307e00640 1 -- 172.21.15.105:0/4101313795 reap_dead start 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f7307400640 1 -- 172.21.15.105:0/4101313795 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec003520 con 0x7f72ec0039c0 2024-03-23T07:57:15.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.647+0000 7f72fea00640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ec0047c0 0x7f72ec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.648+0000 7f7307400640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f72ec003dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c160 tx=0x7f72ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c126a5821be0b431 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.648+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8005ad0 con 0x7f72ec0039c0 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.648+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f8005c30 con 0x7f72ec0039c0 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.648+0000 7f7306a00640 1 -- 172.21.15.105:0/4101313795 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f802c7c0 con 0x7f72ec0039c0 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.649+0000 7f730e800640 1 -- 172.21.15.105:0/4101313795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 msgr2=0x7f72ec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.649+0000 7f730e800640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f72ec003dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c160 tx=0x7f72ec0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.649+0000 7f730e800640 1 -- 172.21.15.105:0/4101313795 shutdown_connections 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.649+0000 7f730e800640 1 --2- 172.21.15.105:0/4101313795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f72ec003dc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.649+0000 7f730e800640 1 -- 172.21.15.105:0/4101313795 >> 172.21.15.105:0/4101313795 conn(0x7f73080cf5d0 msgr2=0x7f73080cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:15.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.649+0000 7f730e800640 1 -- 172.21.15.105:0/4101313795 shutdown_connections 2024-03-23T07:57:15.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.649+0000 7f730e800640 1 -- 172.21.15.105:0/4101313795 wait complete. 2024-03-23T07:57:15.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.651+0000 7f730e800640 1 Processor -- start 2024-03-23T07:57:15.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.651+0000 7f730e800640 1 -- start start 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f730e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f73081f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f730e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73081f5c50 0x7f730815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f730e800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73081f60d0 0x7f730815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f730e800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730814e610 con 0x7f72ec0039c0 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f730e800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73081f1d20 con 0x7f73081f60d0 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f730e800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080b3600 con 0x7f73081f5c50 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f73081f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f73081f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60002/0 (socket says 172.21.15.105:60002) 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 -- 172.21.15.105:0/2635163214 learned_addr learned my addr 172.21.15.105:0/2635163214 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:15.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307e00640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73081f60d0 0x7f730815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 -- 172.21.15.105:0/2635163214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73081f5c50 msgr2=0x7f730815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73081f5c50 0x7f730815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 -- 172.21.15.105:0/2635163214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73081f60d0 msgr2=0x7f730815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73081f60d0 0x7f730815ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.652+0000 7f7307400640 1 -- 172.21.15.105:0/2635163214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730814da90 con 0x7f72ec0039c0 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.653+0000 7f7307400640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f73081f17e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c160 tx=0x7f72f802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.653+0000 7f72ffe00640 1 -- 172.21.15.105:0/2635163214 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f803c040 con 0x7f72ec0039c0 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.653+0000 7f72ffe00640 1 -- 172.21.15.105:0/2635163214 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f8005a40 con 0x7f72ec0039c0 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.653+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730814dd70 con 0x7f72ec0039c0 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.653+0000 7f72ffe00640 1 -- 172.21.15.105:0/2635163214 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8007290 con 0x7f72ec0039c0 2024-03-23T07:57:15.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.653+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730814e330 con 0x7f72ec0039c0 2024-03-23T07:57:15.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.654+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730814b5b0 con 0x7f72ec0039c0 2024-03-23T07:57:15.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.655+0000 7f72ffe00640 1 -- 172.21.15.105:0/2635163214 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f72f8032070 con 0x7f72ec0039c0 2024-03-23T07:57:15.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.656+0000 7f72ffe00640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72e406ac00 0x7f72e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:15.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.657+0000 7f72ffe00640 1 -- 172.21.15.105:0/2635163214 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f72f80aaaf0 con 0x7f72ec0039c0 2024-03-23T07:57:15.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.657+0000 7f72fea00640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72e406ac00 0x7f72e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:15.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.657+0000 7f72fea00640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72e406ac00 0x7f72e406d0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73080b5ef0 tx=0x7f72f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:15.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.661+0000 7f72ffe00640 1 -- 172.21.15.105:0/2635163214 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f72f8075ab0 con 0x7f72ec0039c0 2024-03-23T07:57:15.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.842+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7308073390 con 0x7f72ec0039c0 2024-03-23T07:57:15.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.843+0000 7f72ffe00640 1 -- 172.21.15.105:0/2635163214 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72f8046070 con 0x7f72ec0039c0 2024-03-23T07:57:15.843 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.845+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72e406ac00 msgr2=0x7f72e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72e406ac00 0x7f72e406d0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73080b5ef0 tx=0x7f72f0009040 comp rx=0 tx=0).stop 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 msgr2=0x7f73081f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f73081f17e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c160 tx=0x7f72f802e070 comp rx=0 tx=0).stop 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 shutdown_connections 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73081f60d0 0x7f730815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f72e406ac00 0x7f72e406d0c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73081f5c50 0x7f730815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 --2- 172.21.15.105:0/2635163214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72ec0039c0 0x7f73081f17e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 >> 172.21.15.105:0/2635163214 conn(0x7f73080cf5d0 msgr2=0x7f73080b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 shutdown_connections 2024-03-23T07:57:15.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:15.846+0000 7f730e800640 1 -- 172.21.15.105:0/2635163214 wait complete. 2024-03-23T07:57:15.864 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:16.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.070+0000 7f966ee00640 1 Processor -- start 2024-03-23T07:57:16.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.070+0000 7f966ee00640 1 -- start start 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f96680ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966ee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96680cc200 0x7f96680cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966ee00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96680ad7a0 0x7f96680cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966ee00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96680cb5b0 con 0x7f96680ad7a0 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966ee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96680c8c90 con 0x7f96680cc200 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96680cc6c0 con 0x7f9668159cd0 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966e400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96680ad7a0 0x7f96680cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966e400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96680ad7a0 0x7f96680cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60014/0 (socket says 172.21.15.105:60014) 2024-03-23T07:57:16.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966e400640 1 -- 172.21.15.105:0/2352511039 learned_addr learned my addr 172.21.15.105:0/2352511039 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966d000640 1 --2- 172.21.15.105:0/2352511039 >> v2:172.21.15.132:3300/0 conn(0x7f96680cc200 0x7f96680cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966da00640 1 --2- 172.21.15.105:0/2352511039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f96680ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966e400640 1 -- 172.21.15.105:0/2352511039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 msgr2=0x7f96680ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.072+0000 7f966e400640 1 --2- 172.21.15.105:0/2352511039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f96680ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.073+0000 7f966e400640 1 -- 172.21.15.105:0/2352511039 >> v2:172.21.15.132:3300/0 conn(0x7f96680cc200 msgr2=0x7f96680cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.073+0000 7f966e400640 1 --2- 172.21.15.105:0/2352511039 >> v2:172.21.15.132:3300/0 conn(0x7f96680cc200 0x7f96680cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.073+0000 7f966e400640 1 -- 172.21.15.105:0/2352511039 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96680cc9a0 con 0x7f96680ad7a0 2024-03-23T07:57:16.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.073+0000 7f966da00640 1 --2- 172.21.15.105:0/2352511039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f96680ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:16.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.074+0000 7f966e400640 1 --2- 172.21.15.105:0/2352511039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680cb070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f966806be50 tx=0x7f966002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2737fcaafca5445 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.074+0000 7f9667e00640 1 -- 172.21.15.105:0/2352511039 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966002ca70 con 0x7f96680ad7a0 2024-03-23T07:57:16.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.074+0000 7f9667e00640 1 -- 172.21.15.105:0/2352511039 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966002cbd0 con 0x7f96680ad7a0 2024-03-23T07:57:16.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.074+0000 7f9667e00640 1 -- 172.21.15.105:0/2352511039 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600343f0 con 0x7f96680ad7a0 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.074+0000 7f966ee00640 1 -- 172.21.15.105:0/2352511039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 msgr2=0x7f96680cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.074+0000 7f966ee00640 1 --2- 172.21.15.105:0/2352511039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680cb070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f966806be50 tx=0x7f966002bed0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.075+0000 7f966ee00640 1 -- 172.21.15.105:0/2352511039 shutdown_connections 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.075+0000 7f966ee00640 1 --2- 172.21.15.105:0/2352511039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680cb070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.075+0000 7f966ee00640 1 --2- 172.21.15.105:0/2352511039 >> v2:172.21.15.132:3300/0 conn(0x7f96680cc200 0x7f96680cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.075+0000 7f966ee00640 1 --2- 172.21.15.105:0/2352511039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f96680ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.075+0000 7f966ee00640 1 -- 172.21.15.105:0/2352511039 >> 172.21.15.105:0/2352511039 conn(0x7f96680b01b0 msgr2=0x7f96680b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.075+0000 7f966ee00640 1 -- 172.21.15.105:0/2352511039 shutdown_connections 2024-03-23T07:57:16.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.075+0000 7f966ee00640 1 -- 172.21.15.105:0/2352511039 wait complete. 2024-03-23T07:57:16.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.076+0000 7f966ee00640 1 Processor -- start 2024-03-23T07:57:16.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.076+0000 7f966ee00640 1 -- start start 2024-03-23T07:57:16.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.077+0000 7f966ee00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.077+0000 7f966ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96680cc200 0x7f966809bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f966815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966ee00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96681f3110 con 0x7f96680ad7a0 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966814ef70 con 0x7f96680cc200 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966809c2f0 con 0x7f9668159cd0 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60026/0 (socket says 172.21.15.105:60026) 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 -- 172.21.15.105:0/123126931 learned_addr learned my addr 172.21.15.105:0/123126931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 -- 172.21.15.105:0/123126931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 msgr2=0x7f966815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f966815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 -- 172.21.15.105:0/123126931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96680cc200 msgr2=0x7f966809bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96680cc200 0x7f966809bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 -- 172.21.15.105:0/123126931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660009b30 con 0x7f96680ad7a0 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.078+0000 7f966da00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680c3240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c7f0 tx=0x7f965c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.079+0000 7f9665600640 1 -- 172.21.15.105:0/123126931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c007820 con 0x7f96680ad7a0 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.079+0000 7f9665600640 1 -- 172.21.15.105:0/123126931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c007980 con 0x7f96680ad7a0 2024-03-23T07:57:16.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.079+0000 7f9665600640 1 -- 172.21.15.105:0/123126931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c007cf0 con 0x7f96680ad7a0 2024-03-23T07:57:16.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.079+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966814e2f0 con 0x7f96680ad7a0 2024-03-23T07:57:16.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.079+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966814e8b0 con 0x7f96680ad7a0 2024-03-23T07:57:16.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.080+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966804e030 con 0x7f96680ad7a0 2024-03-23T07:57:16.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.081+0000 7f9665600640 1 -- 172.21.15.105:0/123126931 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f965c0113f0 con 0x7f96680ad7a0 2024-03-23T07:57:16.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.082+0000 7f9665600640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f964806ac00 0x7f964806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.082+0000 7f9665600640 1 -- 172.21.15.105:0/123126931 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f965c0899d0 con 0x7f96680ad7a0 2024-03-23T07:57:16.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.082+0000 7f966d000640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f964806ac00 0x7f964806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.083+0000 7f966d000640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f964806ac00 0x7f964806d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f96680acab0 tx=0x7f9654008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.086+0000 7f9665600640 1 -- 172.21.15.105:0/123126931 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f965c054850 con 0x7f96680ad7a0 2024-03-23T07:57:16.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.282+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96680cdd20 con 0x7f96680ad7a0 2024-03-23T07:57:16.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.283+0000 7f9665600640 1 -- 172.21.15.105:0/123126931 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f965c057e70 con 0x7f96680ad7a0 2024-03-23T07:57:16.283 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:16.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.285+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f964806ac00 msgr2=0x7f964806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.285+0000 7f966ee00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f964806ac00 0x7f964806d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f96680acab0 tx=0x7f9654008040 comp rx=0 tx=0).stop 2024-03-23T07:57:16.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.285+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 msgr2=0x7f96680c3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.285+0000 7f966ee00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680c3240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c7f0 tx=0x7f965c00ccb0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 shutdown_connections 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f964806ac00 0x7f964806d0c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9668159cd0 0x7f966815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96680cc200 0x7f966809bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 --2- 172.21.15.105:0/123126931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96680ad7a0 0x7f96680c3240 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 >> 172.21.15.105:0/123126931 conn(0x7f96680b01b0 msgr2=0x7f96680abb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 shutdown_connections 2024-03-23T07:57:16.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.286+0000 7f966ee00640 1 -- 172.21.15.105:0/123126931 wait complete. 2024-03-23T07:57:16.303 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:16.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.511+0000 7f4162600640 1 Processor -- start 2024-03-23T07:57:16.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.511+0000 7f4162600640 1 -- start start 2024-03-23T07:57:16.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.512+0000 7f4162600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 0x7f415c0add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f4162600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c0cdba0 con 0x7f415c06ce90 2024-03-23T07:57:16.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f4162600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c0cde80 con 0x7f415c0cd3e0 2024-03-23T07:57:16.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f4162600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c06c760 con 0x7f415c0ce930 2024-03-23T07:57:16.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f415be00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f415c06ce90 0x7f415c15e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40504/0 (socket says 172.21.15.105:40504) 2024-03-23T07:57:16.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f415be00640 1 -- 172.21.15.105:0/1466299470 learned_addr learned my addr 172.21.15.105:0/1466299470 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f415b400640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 0x7f415c0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519131840 0 0) 0x7f415c0cdba0 con 0x7f415c06ce90 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.513+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4140003520 con 0x7f415c06ce90 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3499573231 0 0) 0x7f4140003520 con 0x7f415c06ce90 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 msgr2=0x7f415c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 0x7f415c0add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 >> v1:172.21.15.132:6789/0 conn(0x7f415c0cd3e0 legacy=0x7f415c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0ce160 con 0x7f415c06ce90 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f41540019f0 con 0x7f415c06ce90 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 >> v1:172.21.15.105:6789/0 conn(0x7f415c06ce90 legacy=0x7f415c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41400039a0 0x7f4140003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f4140004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 0x7f4140005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c06c760 con 0x7f41400047c0 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414000ba60 con 0x7f41400039a0 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414000bc20 con 0x7f415c0ce930 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415aa00640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f4140004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.514+0000 7f415b400640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41400039a0 0x7f4140003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.515+0000 7f415be00640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 0x7f4140005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.515+0000 7f415aa00640 1 -- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 msgr2=0x7f4140005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.515+0000 7f415aa00640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c0ce930 0x7f4140005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.515+0000 7f415aa00640 1 -- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41400039a0 msgr2=0x7f4140003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.515+0000 7f415aa00640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41400039a0 0x7f4140003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.515+0000 7f415aa00640 1 -- 172.21.15.105:0/1466299470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4140003520 con 0x7f41400047c0 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.516+0000 7f415aa00640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f4140004c40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4148001720 tx=0x7f41400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1ec8a46b92ef0b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.516+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41480052d0 con 0x7f41400047c0 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.516+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414802e050 con 0x7f41400047c0 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.516+0000 7f415a000640 1 -- 172.21.15.105:0/1466299470 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41480035f0 con 0x7f41400047c0 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.516+0000 7f4162600640 1 -- 172.21.15.105:0/1466299470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 msgr2=0x7f4140004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.516+0000 7f4162600640 1 --2- 172.21.15.105:0/1466299470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f4140004c40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4148001720 tx=0x7f41400057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.516+0000 7f415be00640 1 -- 172.21.15.105:0/1466299470 reap_dead start 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.517+0000 7f4162600640 1 -- 172.21.15.105:0/1466299470 shutdown_connections 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.517+0000 7f4162600640 1 -- 172.21.15.105:0/1466299470 >> 172.21.15.105:0/1466299470 conn(0x7f415c06a0b0 msgr2=0x7f415c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.517+0000 7f4162600640 1 -- 172.21.15.105:0/1466299470 shutdown_connections 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.517+0000 7f4162600640 1 -- 172.21.15.105:0/1466299470 wait complete. 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.518+0000 7f4162600640 1 Processor -- start 2024-03-23T07:57:16.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.519+0000 7f4162600640 1 -- start start 2024-03-23T07:57:16.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.519+0000 7f4162600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f415c153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.519+0000 7f4162600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c140870 0x7f415c0af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.519+0000 7f4162600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f415c0af620 0x7f415c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.519+0000 7f4162600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c154020 con 0x7f41400047c0 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.519+0000 7f4162600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0adc90 con 0x7f415c0af620 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.519+0000 7f4162600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0ad710 con 0x7f415c140870 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f415c153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f415c153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60032/0 (socket says 172.21.15.105:60032) 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 -- 172.21.15.105:0/1424444292 learned_addr learned my addr 172.21.15.105:0/1424444292 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 -- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c140870 msgr2=0x7f415c0af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c140870 0x7f415c0af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 -- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f415c0af620 msgr2=0x7f415c0afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415be00640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f415c0af620 0x7f415c0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f415c0af620 0x7f415c0afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 -- 172.21.15.105:0/1424444292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0affe0 con 0x7f41400047c0 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415be00640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f415c0af620 0x7f415c0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.520+0000 7f415b400640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f415c153ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f414c011630 tx=0x7f414c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.521+0000 7f4153400640 1 -- 172.21.15.105:0/1424444292 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c00f920 con 0x7f41400047c0 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.521+0000 7f4153400640 1 -- 172.21.15.105:0/1424444292 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c00fa80 con 0x7f41400047c0 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.521+0000 7f4153400640 1 -- 172.21.15.105:0/1424444292 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c00fdf0 con 0x7f41400047c0 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.521+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c13dbb0 con 0x7f41400047c0 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.521+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c154360 con 0x7f41400047c0 2024-03-23T07:57:16.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.522+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c1541c0 con 0x7f41400047c0 2024-03-23T07:57:16.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.523+0000 7f4153400640 1 -- 172.21.15.105:0/1424444292 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f414c015460 con 0x7f41400047c0 2024-03-23T07:57:16.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.524+0000 7f4153400640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f413c06ac00 0x7f413c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.524+0000 7f4153400640 1 -- 172.21.15.105:0/1424444292 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f414c089b40 con 0x7f41400047c0 2024-03-23T07:57:16.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.524+0000 7f415aa00640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f413c06ac00 0x7f413c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.525+0000 7f415aa00640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f413c06ac00 0x7f413c06d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4148033000 tx=0x7f414802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.528+0000 7f4153400640 1 -- 172.21.15.105:0/1424444292 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f415c1541c0 con 0x7f41400047c0 2024-03-23T07:57:16.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.721+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f415c1541c0 con 0x7f41400047c0 2024-03-23T07:57:16.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.722+0000 7f4153400640 1 -- 172.21.15.105:0/1424444292 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f415c1541c0 con 0x7f41400047c0 2024-03-23T07:57:16.722 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f413c06ac00 msgr2=0x7f413c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f413c06ac00 0x7f413c06d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4148033000 tx=0x7f414802f000 comp rx=0 tx=0).stop 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 msgr2=0x7f415c153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f415c153ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f414c011630 tx=0x7f414c011af0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 shutdown_connections 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f415c0af620 0x7f415c0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f413c06ac00 0x7f413c06d0c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f415c140870 0x7f415c0af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 --2- 172.21.15.105:0/1424444292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400047c0 0x7f415c153ae0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 >> 172.21.15.105:0/1424444292 conn(0x7f415c06a0b0 msgr2=0x7f415c15bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:16.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.725+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 shutdown_connections 2024-03-23T07:57:16.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.726+0000 7f4162600640 1 -- 172.21.15.105:0/1424444292 wait complete. 2024-03-23T07:57:16.744 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.957+0000 7f7545800640 1 Processor -- start 2024-03-23T07:57:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.957+0000 7f7545800640 1 -- start start 2024-03-23T07:57:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.959+0000 7f7545800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75400b84b0 0x7f75400d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.959+0000 7f7545800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75400d1280 con 0x7f75400b5aa0 2024-03-23T07:57:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.959+0000 7f7545800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75400d20c0 con 0x7f75400cd740 2024-03-23T07:57:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.959+0000 7f7545800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75400b8910 con 0x7f75400b84b0 2024-03-23T07:57:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.959+0000 7f753f400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f75400b5aa0 0x7f75400b5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47706/0 (socket says 172.21.15.105:47706) 2024-03-23T07:57:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.959+0000 7f753f400640 1 -- 172.21.15.105:0/2139853466 learned_addr learned my addr 172.21.15.105:0/2139853466 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753e000640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75400b84b0 0x7f75400d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781822302 0 0) 0x7f75400d1280 con 0x7f75400b5aa0 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7524003520 con 0x7f75400b5aa0 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2814063671 0 0) 0x7f7524003520 con 0x7f75400b5aa0 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75400b84b0 msgr2=0x7f75400d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75400b84b0 0x7f75400d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 >> v1:172.21.15.132:6789/0 conn(0x7f75400cd740 legacy=0x7f75400cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400d14b0 con 0x7f75400b5aa0 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f7538001670 con 0x7f75400b5aa0 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 >> v1:172.21.15.105:6789/0 conn(0x7f75400b5aa0 legacy=0x7f75400b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75240039a0 0x7f7524003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75240047c0 0x7f7524004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7524005180 0x7f752400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400b8910 con 0x7f7524005180 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752400c080 con 0x7f75240047c0 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.960+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752400c240 con 0x7f75240039a0 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753f400640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7524005180 0x7f752400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753e000640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75240047c0 0x7f7524004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753ea00640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75240039a0 0x7f7524003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753f400640 1 -- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75240039a0 msgr2=0x7f7524003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753f400640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75240039a0 0x7f7524003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753f400640 1 -- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75240047c0 msgr2=0x7f7524004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753f400640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75240047c0 0x7f7524004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.961+0000 7f753f400640 1 -- 172.21.15.105:0/2139853466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524003520 con 0x7f7524005180 2024-03-23T07:57:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.962+0000 7f753f400640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7524005180 0x7f752400b980 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f753800c0e0 tx=0x7f75240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956654378caf2355 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.962+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538005ad0 con 0x7f7524005180 2024-03-23T07:57:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.962+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7538005c30 con 0x7f7524005180 2024-03-23T07:57:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.962+0000 7f753d600640 1 -- 172.21.15.105:0/2139853466 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753802ca50 con 0x7f7524005180 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.963+0000 7f7545800640 1 -- 172.21.15.105:0/2139853466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7524005180 msgr2=0x7f752400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.963+0000 7f7545800640 1 --2- 172.21.15.105:0/2139853466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7524005180 0x7f752400b980 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f753800c0e0 tx=0x7f75240057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.963+0000 7f753f400640 1 -- 172.21.15.105:0/2139853466 reap_dead start 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.963+0000 7f7545800640 1 -- 172.21.15.105:0/2139853466 shutdown_connections 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.963+0000 7f7545800640 1 -- 172.21.15.105:0/2139853466 >> 172.21.15.105:0/2139853466 conn(0x7f75400cf5d0 msgr2=0x7f75400cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.963+0000 7f7545800640 1 -- 172.21.15.105:0/2139853466 shutdown_connections 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.963+0000 7f7545800640 1 -- 172.21.15.105:0/2139853466 wait complete. 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.965+0000 7f7545800640 1 Processor -- start 2024-03-23T07:57:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.965+0000 7f7545800640 1 -- start start 2024-03-23T07:57:16.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f7545800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7524005180 0x7f75400c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f7545800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f754014e870 0x7f75400b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f7545800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7540148d20 0x7f754014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f7545800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400cc2d0 con 0x7f754014e870 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f7545800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400b2dc0 con 0x7f7540148d20 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f7545800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754014bb30 con 0x7f7524005180 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f753e000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f754014e870 0x7f75400b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f753e000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f754014e870 0x7f75400b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36032/0 (socket says 172.21.15.105:36032) 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.966+0000 7f753e000640 1 -- 172.21.15.105:0/2396329627 learned_addr learned my addr 172.21.15.105:0/2396329627 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753ea00640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7524005180 0x7f75400c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753f400640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7540148d20 0x7f754014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753e000640 1 -- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7524005180 msgr2=0x7f75400c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753e000640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7524005180 0x7f75400c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753e000640 1 -- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7540148d20 msgr2=0x7f754014adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753e000640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7540148d20 0x7f754014adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753e000640 1 -- 172.21.15.105:0/2396329627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400cc4e0 con 0x7f754014e870 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753f400640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7540148d20 0x7f754014adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f753e000640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f754014e870 0x7f75400b13e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f752c009c60 tx=0x7f752c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f7536a00640 1 -- 172.21.15.105:0/2396329627 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c003f20 con 0x7f754014e870 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f7536a00640 1 -- 172.21.15.105:0/2396329627 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c00f050 con 0x7f754014e870 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75400cc7c0 con 0x7f754014e870 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.967+0000 7f7536a00640 1 -- 172.21.15.105:0/2396329627 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c014830 con 0x7f754014e870 2024-03-23T07:57:16.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.968+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754014b2f0 con 0x7f754014e870 2024-03-23T07:57:16.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.969+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7540073390 con 0x7f754014e870 2024-03-23T07:57:16.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.970+0000 7f7536a00640 1 -- 172.21.15.105:0/2396329627 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f752c014990 con 0x7f754014e870 2024-03-23T07:57:16.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.971+0000 7f7536a00640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f752006ac00 0x7f752006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:16.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.971+0000 7f7536a00640 1 -- 172.21.15.105:0/2396329627 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f752c089fc0 con 0x7f754014e870 2024-03-23T07:57:16.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.971+0000 7f753ea00640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f752006ac00 0x7f752006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:16.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.972+0000 7f753ea00640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f752006ac00 0x7f752006d0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7530000c00 tx=0x7f7530009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:16.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:16.975+0000 7f7536a00640 1 -- 172.21.15.105:0/2396329627 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f752c054db0 con 0x7f754014e870 2024-03-23T07:57:17.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.171+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f75400b2f00 con 0x7f754014e870 2024-03-23T07:57:17.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.172+0000 7f7536a00640 1 -- 172.21.15.105:0/2396329627 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f75400b2f00 con 0x7f754014e870 2024-03-23T07:57:17.172 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f752006ac00 msgr2=0x7f752006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f752006ac00 0x7f752006d0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7530000c00 tx=0x7f7530009040 comp rx=0 tx=0).stop 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f754014e870 msgr2=0x7f75400b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f754014e870 0x7f75400b13e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f752c009c60 tx=0x7f752c007700 comp rx=0 tx=0).stop 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 shutdown_connections 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7540148d20 0x7f754014adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f754014e870 0x7f75400b13e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f752006ac00 0x7f752006d0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 --2- 172.21.15.105:0/2396329627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7524005180 0x7f75400c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 >> 172.21.15.105:0/2396329627 conn(0x7f75400cf5d0 msgr2=0x7f75400cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 shutdown_connections 2024-03-23T07:57:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.175+0000 7f7545800640 1 -- 172.21.15.105:0/2396329627 wait complete. 2024-03-23T07:57:17.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.387+0000 7f01ee400640 1 Processor -- start 2024-03-23T07:57:17.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.387+0000 7f01ee400640 1 -- start start 2024-03-23T07:57:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.389+0000 7f01ee400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 0x7f01e80b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.389+0000 7f01ee400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e806c6f0 con 0x7f01e815c400 2024-03-23T07:57:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.389+0000 7f01ee400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e806c9d0 con 0x7f01e815b560 2024-03-23T07:57:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.389+0000 7f01ee400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80b4370 con 0x7f01e80b39b0 2024-03-23T07:57:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.389+0000 7f01e7e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f01e815c400 0x7f01e815c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47716/0 (socket says 172.21.15.105:47716) 2024-03-23T07:57:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.389+0000 7f01e7e00640 1 -- 172.21.15.105:0/2110403631 learned_addr learned my addr 172.21.15.105:0/2110403631 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.389+0000 7f01dea00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 0x7f01e80b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322641031 0 0) 0x7f01e806c6f0 con 0x7f01e815c400 2024-03-23T07:57:17.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01cc003520 con 0x7f01e815c400 2024-03-23T07:57:17.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1351501599 0 0) 0x7f01cc003520 con 0x7f01e815c400 2024-03-23T07:57:17.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 msgr2=0x7f01e80b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:17.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 0x7f01e80b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 >> v1:172.21.15.132:6789/0 conn(0x7f01e815b560 legacy=0x7f01e815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e806ccb0 con 0x7f01e815c400 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f01e00019f0 con 0x7f01e815c400 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 >> v1:172.21.15.105:6789/0 conn(0x7f01e815c400 legacy=0x7f01e815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 0x7f01cc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01cc003f70 0x7f01cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc004d00 0x7f01cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80b4370 con 0x7f01cc003f70 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc00bac0 con 0x7f01cc004d00 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc00bc80 con 0x7f01e80b39b0 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.390+0000 7f01dea00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01cc003f70 0x7f01cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01e7e00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc004d00 0x7f01cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01e7400640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 0x7f01cc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01dea00640 1 -- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 msgr2=0x7f01cc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01dea00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 0x7f01cc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01dea00640 1 -- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc004d00 msgr2=0x7f01cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01dea00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc004d00 0x7f01cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01dea00640 1 -- 172.21.15.105:0/2110403631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc003520 con 0x7f01cc003f70 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.391+0000 7f01e7400640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e80b39b0 0x7f01cc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.392+0000 7f01dea00640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01cc003f70 0x7f01cc0047c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009ac0 tx=0x7f01cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698517a99b9647fa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.392+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d802c600 con 0x7f01cc003f70 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.392+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d802c760 con 0x7f01cc003f70 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.392+0000 7f01e6a00640 1 -- 172.21.15.105:0/2110403631 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d802caf0 con 0x7f01cc003f70 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.393+0000 7f01ee400640 1 -- 172.21.15.105:0/2110403631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01cc003f70 msgr2=0x7f01cc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.393+0000 7f01ee400640 1 --2- 172.21.15.105:0/2110403631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01cc003f70 0x7f01cc0047c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009ac0 tx=0x7f01cc0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.393+0000 7f01e7e00640 1 -- 172.21.15.105:0/2110403631 reap_dead start 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.393+0000 7f01ee400640 1 -- 172.21.15.105:0/2110403631 shutdown_connections 2024-03-23T07:57:17.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.393+0000 7f01ee400640 1 -- 172.21.15.105:0/2110403631 >> 172.21.15.105:0/2110403631 conn(0x7f01e806a3d0 msgr2=0x7f01e80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:17.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.393+0000 7f01ee400640 1 -- 172.21.15.105:0/2110403631 shutdown_connections 2024-03-23T07:57:17.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.393+0000 7f01ee400640 1 -- 172.21.15.105:0/2110403631 wait complete. 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.395+0000 7f01ee400640 1 Processor -- start 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.395+0000 7f01ee400640 1 -- start start 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01ee400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc003f70 0x7f01e813df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01ee400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01e8151fa0 0x7f01e8152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01ee400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e8152420 0x7f01e81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01ee400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e813e4a0 con 0x7f01e8151fa0 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01ee400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e814e6f0 con 0x7f01cc003f70 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01ee400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80cba90 con 0x7f01e8152420 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01e8151fa0 0x7f01e8152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01e8151fa0 0x7f01e8152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36064/0 (socket says 172.21.15.105:36064) 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 -- 172.21.15.105:0/2836766646 learned_addr learned my addr 172.21.15.105:0/2836766646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01e7400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc003f70 0x7f01e813df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01e7e00640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e8152420 0x7f01e81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 -- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e8152420 msgr2=0x7f01e81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e8152420 0x7f01e81531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 -- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc003f70 msgr2=0x7f01e813df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc003f70 0x7f01e813df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.396+0000 7f01dea00640 1 -- 172.21.15.105:0/2836766646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80cbd70 con 0x7f01e8151fa0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.397+0000 7f01dea00640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01e8151fa0 0x7f01e8152c60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f01d802c1a0 tx=0x7f01d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.397+0000 7f01dfe00640 1 -- 172.21.15.105:0/2836766646 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8003810 con 0x7f01e8151fa0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.397+0000 7f01dfe00640 1 -- 172.21.15.105:0/2836766646 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8003970 con 0x7f01e8151fa0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.397+0000 7f01dfe00640 1 -- 172.21.15.105:0/2836766646 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8003ce0 con 0x7f01e8151fa0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.397+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e80cc050 con 0x7f01e8151fa0 2024-03-23T07:57:17.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.397+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e80cc610 con 0x7f01e8151fa0 2024-03-23T07:57:17.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.398+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e8151e10 con 0x7f01e8151fa0 2024-03-23T07:57:17.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.399+0000 7f01dfe00640 1 -- 172.21.15.105:0/2836766646 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f01d8034440 con 0x7f01e8151fa0 2024-03-23T07:57:17.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.400+0000 7f01dfe00640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f01c406ac00 0x7f01c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.400+0000 7f01dfe00640 1 -- 172.21.15.105:0/2836766646 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f01d80ab670 con 0x7f01e8151fa0 2024-03-23T07:57:17.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.400+0000 7f01e7400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f01c406ac00 0x7f01c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.401+0000 7f01e7400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f01c406ac00 0x7f01c406d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f01d0001750 tx=0x7f01d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:17.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.404+0000 7f01dfe00640 1 -- 172.21.15.105:0/2836766646 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f01d8076580 con 0x7f01e8151fa0 2024-03-23T07:57:17.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.603+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01e806ac20 con 0x7f01e8151fa0 2024-03-23T07:57:17.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.604+0000 7f01dfe00640 1 -- 172.21.15.105:0/2836766646 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f01d80044e0 con 0x7f01e8151fa0 2024-03-23T07:57:17.604 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:17.604 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f01c406ac00 msgr2=0x7f01c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f01c406ac00 0x7f01c406d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f01d0001750 tx=0x7f01d0009040 comp rx=0 tx=0).stop 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01e8151fa0 msgr2=0x7f01e8152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01e8151fa0 0x7f01e8152c60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f01d802c1a0 tx=0x7f01d8004730 comp rx=0 tx=0).stop 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 shutdown_connections 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f01c406ac00 0x7f01c406d0c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01e8152420 0x7f01e81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01e8151fa0 0x7f01e8152c60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 --2- 172.21.15.105:0/2836766646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01cc003f70 0x7f01e813df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 >> 172.21.15.105:0/2836766646 conn(0x7f01e806a3d0 msgr2=0x7f01e80b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:17.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 shutdown_connections 2024-03-23T07:57:17.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.607+0000 7f01ee400640 1 -- 172.21.15.105:0/2836766646 wait complete. 2024-03-23T07:57:17.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:17.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.806+0000 7f5048400640 1 Processor -- start 2024-03-23T07:57:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.806+0000 7f5048400640 1 -- start start 2024-03-23T07:57:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.807+0000 7f5048400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f50400b84b0 0x7f50400d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f5048400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400d1280 con 0x7f50400cd740 2024-03-23T07:57:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f5048400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400d20c0 con 0x7f50400b5aa0 2024-03-23T07:57:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f5048400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50400b8940 con 0x7f50400b84b0 2024-03-23T07:57:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f5045a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f50400cd740 0x7f50400cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47726/0 (socket says 172.21.15.105:47726) 2024-03-23T07:57:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f5045a00640 1 -- 172.21.15.105:0/3778380727 learned_addr learned my addr 172.21.15.105:0/3778380727 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888097489 0 0) 0x7f50400d1280 con 0x7f50400cd740 2024-03-23T07:57:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5024003520 con 0x7f50400cd740 2024-03-23T07:57:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f5045000640 1 --2- 172.21.15.105:0/3778380727 >> v2:172.21.15.132:3300/0 conn(0x7f50400b84b0 0x7f50400d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718083586 0 0) 0x7f5024003520 con 0x7f50400cd740 2024-03-23T07:57:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 >> v2:172.21.15.132:3300/0 conn(0x7f50400b84b0 msgr2=0x7f50400d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f503fe00640 1 --2- 172.21.15.105:0/3778380727 >> v2:172.21.15.132:3300/0 conn(0x7f50400b84b0 0x7f50400d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 >> v1:172.21.15.132:6789/0 conn(0x7f50400b5aa0 legacy=0x7f50400b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.808+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400d14b0 con 0x7f50400cd740 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f5030001670 con 0x7f50400cd740 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 >> v1:172.21.15.105:6789/0 conn(0x7f50400cd740 legacy=0x7f50400cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5024003970 0x7f5024003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50240047c0 0x7f5024004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5024005180 0x7f502400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400b8940 con 0x7f50240047c0 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c0b0 con 0x7f5024005180 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c270 con 0x7f5024003970 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5045000640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50240047c0 0x7f5024004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5045000640 1 -- 172.21.15.105:0/3778380727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5024003970 msgr2=0x7f5024003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5045000640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5024003970 0x7f5024003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5045000640 1 -- 172.21.15.105:0/3778380727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5024005180 msgr2=0x7f502400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5046400640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5024005180 0x7f502400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5045000640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5024005180 0x7f502400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5046400640 1 -- 172.21.15.105:0/3778380727 reap_dead start 2024-03-23T07:57:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.809+0000 7f5045000640 1 -- 172.21.15.105:0/3778380727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024003520 con 0x7f50240047c0 2024-03-23T07:57:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.810+0000 7f5045000640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50240047c0 0x7f5024004c40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f502c00ba00 tx=0x7f50240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25f4f0034fc89be server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c032620 con 0x7f50240047c0 2024-03-23T07:57:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c032780 con 0x7f50240047c0 2024-03-23T07:57:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f503fe00640 1 -- 172.21.15.105:0/3778380727 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c032b10 con 0x7f50240047c0 2024-03-23T07:57:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f5048400640 1 -- 172.21.15.105:0/3778380727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50240047c0 msgr2=0x7f5024004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f5048400640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50240047c0 0x7f5024004c40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f502c00ba00 tx=0x7f50240057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f5048400640 1 -- 172.21.15.105:0/3778380727 shutdown_connections 2024-03-23T07:57:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f5048400640 1 --2- 172.21.15.105:0/3778380727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50240047c0 0x7f5024004c40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f5048400640 1 -- 172.21.15.105:0/3778380727 >> 172.21.15.105:0/3778380727 conn(0x7f50400cf5d0 msgr2=0x7f50400cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f5048400640 1 -- 172.21.15.105:0/3778380727 shutdown_connections 2024-03-23T07:57:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.811+0000 7f5048400640 1 -- 172.21.15.105:0/3778380727 wait complete. 2024-03-23T07:57:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.813+0000 7f5048400640 1 Processor -- start 2024-03-23T07:57:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.814+0000 7f5048400640 1 -- start start 2024-03-23T07:57:17.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5048400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50240047c0 0x7f504014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5048400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50400a9270 0x7f50401f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5048400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50400b3320 0x7f50400b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5048400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400b3ce0 con 0x7f50400b3320 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5048400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400a99f0 con 0x7f50400a9270 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5048400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50400a9cd0 con 0x7f50240047c0 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50400b3320 0x7f50400b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50400b3320 0x7f50400b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36070/0 (socket says 172.21.15.105:36070) 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 -- 172.21.15.105:0/3292253526 learned_addr learned my addr 172.21.15.105:0/3292253526 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5045a00640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50240047c0 0x7f504014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5045000640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50400a9270 0x7f50401f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 -- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50240047c0 msgr2=0x7f504014d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50240047c0 0x7f504014d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 -- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50400a9270 msgr2=0x7f50401f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50400a9270 0x7f50401f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:17.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.815+0000 7f5046400640 1 -- 172.21.15.105:0/3292253526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400a9fb0 con 0x7f50400b3320 2024-03-23T07:57:17.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.816+0000 7f5046400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50400b3320 0x7f50400b37a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f504008af40 tx=0x7f503800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:17.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.816+0000 7f503d600640 1 -- 172.21.15.105:0/3292253526 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038012800 con 0x7f50400b3320 2024-03-23T07:57:17.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.816+0000 7f503d600640 1 -- 172.21.15.105:0/3292253526 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5038012960 con 0x7f50400b3320 2024-03-23T07:57:17.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.816+0000 7f503d600640 1 -- 172.21.15.105:0/3292253526 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038012d10 con 0x7f50400b3320 2024-03-23T07:57:17.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.816+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50400aa290 con 0x7f50400b3320 2024-03-23T07:57:17.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.816+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50400a96f0 con 0x7f50400b3320 2024-03-23T07:57:17.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.817+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5040073390 con 0x7f50400b3320 2024-03-23T07:57:17.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.819+0000 7f503d600640 1 -- 172.21.15.105:0/3292253526 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f503801e460 con 0x7f50400b3320 2024-03-23T07:57:17.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.819+0000 7f503d600640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f502006ac30 0x7f502006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:17.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.820+0000 7f503d600640 1 -- 172.21.15.105:0/3292253526 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f50380894a0 con 0x7f50400b3320 2024-03-23T07:57:17.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.820+0000 7f5045a00640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f502006ac30 0x7f502006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:17.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.820+0000 7f5045a00640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f502006ac30 0x7f502006d0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5030000fb0 tx=0x7f5030001600 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:17.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:17.828+0000 7f503d600640 1 -- 172.21.15.105:0/3292253526 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5038054290 con 0x7f50400b3320 2024-03-23T07:57:18.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.026+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50400b20b0 con 0x7f50400b3320 2024-03-23T07:57:18.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.027+0000 7f503d600640 1 -- 172.21.15.105:0/3292253526 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50380578b0 con 0x7f50400b3320 2024-03-23T07:57:18.026 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:18.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.029+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f502006ac30 msgr2=0x7f502006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.029+0000 7f5048400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f502006ac30 0x7f502006d0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5030000fb0 tx=0x7f5030001600 comp rx=0 tx=0).stop 2024-03-23T07:57:18.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.029+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50400b3320 msgr2=0x7f50400b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.029+0000 7f5048400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50400b3320 0x7f50400b37a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f504008af40 tx=0x7f503800d960 comp rx=0 tx=0).stop 2024-03-23T07:57:18.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 shutdown_connections 2024-03-23T07:57:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50400b3320 0x7f50400b37a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50400a9270 0x7f50401f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f502006ac30 0x7f502006d0f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 --2- 172.21.15.105:0/3292253526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50240047c0 0x7f504014d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 >> 172.21.15.105:0/3292253526 conn(0x7f50400cf5d0 msgr2=0x7f504006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 shutdown_connections 2024-03-23T07:57:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.030+0000 7f5048400640 1 -- 172.21.15.105:0/3292253526 wait complete. 2024-03-23T07:57:18.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:18.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.235+0000 7f0ab9a00640 1 Processor -- start 2024-03-23T07:57:18.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.236+0000 7f0ab9a00640 1 -- start start 2024-03-23T07:57:18.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0ab9a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ab415c820 0x7f0ab40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0ab9a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0ab40cf7c0 0x7f0ab40d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0ab9a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab40ccd90 con 0x7f0ab4160e90 2024-03-23T07:57:18.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0ab9a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40cdd70 con 0x7f0ab415c820 2024-03-23T07:57:18.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0ab9a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40b7fc0 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0ab3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ab415c820 0x7f0ab40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0aabe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0ab4160e90 0x7f0ab40cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47738/0 (socket says 172.21.15.105:47738) 2024-03-23T07:57:18.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.237+0000 7f0aabe00640 1 -- 172.21.15.105:0/1199594312 learned_addr learned my addr 172.21.15.105:0/1199594312 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:18.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.238+0000 7f0ab3e00640 1 -- 172.21.15.105:0/1199594312 >> v2:172.21.15.132:3300/0 conn(0x7f0ab40cf7c0 msgr2=0x7f0ab40d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:18.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.238+0000 7f0ab3e00640 1 --2- 172.21.15.105:0/1199594312 >> v2:172.21.15.132:3300/0 conn(0x7f0ab40cf7c0 0x7f0ab40d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.238+0000 7f0ab3e00640 1 -- 172.21.15.105:0/1199594312 >> v1:172.21.15.105:6789/0 conn(0x7f0ab4160e90 legacy=0x7f0ab40cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.238+0000 7f0ab3e00640 1 -- 172.21.15.105:0/1199594312 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40b82a0 con 0x7f0ab415c820 2024-03-23T07:57:18.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.238+0000 7f0ab3400640 1 -- 172.21.15.105:0/1199594312 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079338043 0 0) 0x7f0ab40ccd90 con 0x7f0ab4160e90 2024-03-23T07:57:18.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.239+0000 7f0ab3e00640 1 --2- 172.21.15.105:0/1199594312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab415c820 0x7f0ab40cd830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0ab408b310 tx=0x7f0aa402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec26d5f822e3c8e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:18.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.239+0000 7f0ab3400640 1 -- 172.21.15.105:0/1199594312 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4030610 con 0x7f0ab415c820 2024-03-23T07:57:18.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.239+0000 7f0ab3400640 1 -- 172.21.15.105:0/1199594312 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa4030770 con 0x7f0ab415c820 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.239+0000 7f0ab3400640 1 -- 172.21.15.105:0/1199594312 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4030b00 con 0x7f0ab415c820 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 -- 172.21.15.105:0/1199594312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab415c820 msgr2=0x7f0ab40cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/1199594312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab415c820 0x7f0ab40cd830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0ab408b310 tx=0x7f0aa402b960 comp rx=0 tx=0).stop 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 -- 172.21.15.105:0/1199594312 shutdown_connections 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/1199594312 >> v2:172.21.15.132:3300/0 conn(0x7f0ab40cf7c0 0x7f0ab40d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/1199594312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab415c820 0x7f0ab40cd830 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 -- 172.21.15.105:0/1199594312 >> 172.21.15.105:0/1199594312 conn(0x7f0ab406b050 msgr2=0x7f0ab406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 -- 172.21.15.105:0/1199594312 shutdown_connections 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.240+0000 7f0ab9a00640 1 -- 172.21.15.105:0/1199594312 wait complete. 2024-03-23T07:57:18.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.242+0000 7f0ab9a00640 1 Processor -- start 2024-03-23T07:57:18.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.242+0000 7f0ab9a00640 1 -- start start 2024-03-23T07:57:18.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab9a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab40cf7c0 0x7f0ab41553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab415c820 0x7f0ab4152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab9a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab4160e90 0x7f0ab40c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab9a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41578d0 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab9a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4157bb0 con 0x7f0ab4160e90 2024-03-23T07:57:18.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41548c0 con 0x7f0ab415c820 2024-03-23T07:57:18.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab40cf7c0 0x7f0ab41553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab40cf7c0 0x7f0ab41553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36090/0 (socket says 172.21.15.105:36090) 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 -- 172.21.15.105:0/2774375840 learned_addr learned my addr 172.21.15.105:0/2774375840 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0aabe00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab415c820 0x7f0ab4152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab9000640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab4160e90 0x7f0ab40c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 -- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab415c820 msgr2=0x7f0ab4152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab415c820 0x7f0ab4152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 -- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab4160e90 msgr2=0x7f0ab40c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab4160e90 0x7f0ab40c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.243+0000 7f0ab3e00640 1 -- 172.21.15.105:0/2774375840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4009400 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.244+0000 7f0ab3e00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab40cf7c0 0x7f0ab41553a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009890 tx=0x7f0aa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:18.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.244+0000 7f0ab0c00640 1 -- 172.21.15.105:0/2774375840 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4039c90 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.244+0000 7f0ab0c00640 1 -- 172.21.15.105:0/2774375840 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa403f040 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.244+0000 7f0ab0c00640 1 -- 172.21.15.105:0/2774375840 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4006e10 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.244+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab4154ba0 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.244+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab41558e0 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.245+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab4154720 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.246+0000 7f0ab0c00640 1 -- 172.21.15.105:0/2774375840 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0aa4004030 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.247+0000 7f0ab0c00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0a9006ac00 0x7f0a9006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.247+0000 7f0ab0c00640 1 -- 172.21.15.105:0/2774375840 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0aa40ab9c0 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.247+0000 7f0aabe00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0a9006ac00 0x7f0a9006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.248+0000 7f0aabe00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0a9006ac00 0x7f0a9006d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ab415c3a0 tx=0x7f0a9c007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:18.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.252+0000 7f0ab0c00640 1 -- 172.21.15.105:0/2774375840 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0ab4154720 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.443+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ab4154720 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.444+0000 7f0ab0c00640 1 -- 172.21.15.105:0/2774375840 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0aa4048020 con 0x7f0ab40cf7c0 2024-03-23T07:57:18.443 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:18.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0a9006ac00 msgr2=0x7f0a9006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0a9006ac00 0x7f0a9006d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ab415c3a0 tx=0x7f0a9c007b50 comp rx=0 tx=0).stop 2024-03-23T07:57:18.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab40cf7c0 msgr2=0x7f0ab41553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab40cf7c0 0x7f0ab41553a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009890 tx=0x7f0aa40046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 shutdown_connections 2024-03-23T07:57:18.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab4160e90 0x7f0ab40c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0a9006ac00 0x7f0a9006d0c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab415c820 0x7f0ab4152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.446+0000 7f0ab9a00640 1 --2- 172.21.15.105:0/2774375840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ab40cf7c0 0x7f0ab41553a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.447+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 >> 172.21.15.105:0/2774375840 conn(0x7f0ab406b050 msgr2=0x7f0ab40b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:18.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.447+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 shutdown_connections 2024-03-23T07:57:18.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.447+0000 7f0ab9a00640 1 -- 172.21.15.105:0/2774375840 wait complete. 2024-03-23T07:57:18.464 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:18.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.662+0000 7fbfa1000640 1 Processor -- start 2024-03-23T07:57:18.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.662+0000 7fbfa1000640 1 -- start start 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.663+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0b84b0 0x7fbf9c0d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.663+0000 7fbfa1000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf9c0b5aa0 0x7fbf9c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.663+0000 7fbfa1000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0d1280 con 0x7fbf9c0cd740 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.663+0000 7fbfa1000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0d20c0 con 0x7fbf9c0b5aa0 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.663+0000 7fbfa1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0b8910 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.663+0000 7fbf99e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbf9c0cd740 0x7fbf9c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47752/0 (socket says 172.21.15.105:47752) 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.663+0000 7fbf99e00640 1 -- 172.21.15.105:0/1284628877 learned_addr learned my addr 172.21.15.105:0/1284628877 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:18.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf9a800640 1 --2- 172.21.15.105:0/1284628877 >> v2:172.21.15.105:3300/0 conn(0x7fbf9c0b5aa0 0x7fbf9c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf99400640 1 -- 172.21.15.105:0/1284628877 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927824085 0 0) 0x7fbf9c0d1280 con 0x7fbf9c0cd740 2024-03-23T07:57:18.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf99400640 1 -- 172.21.15.105:0/1284628877 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf80003520 con 0x7fbf9c0cd740 2024-03-23T07:57:18.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf93e00640 1 --2- 172.21.15.105:0/1284628877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0b84b0 0x7fbf9c0d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf9a800640 1 -- 172.21.15.105:0/1284628877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0b84b0 msgr2=0x7fbf9c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:18.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf9a800640 1 --2- 172.21.15.105:0/1284628877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0b84b0 0x7fbf9c0d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf99400640 1 -- 172.21.15.105:0/1284628877 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172105416 0 0) 0x7fbf80003520 con 0x7fbf9c0cd740 2024-03-23T07:57:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf9a800640 1 -- 172.21.15.105:0/1284628877 >> v1:172.21.15.105:6789/0 conn(0x7fbf9c0cd740 legacy=0x7fbf9c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.664+0000 7fbf9a800640 1 -- 172.21.15.105:0/1284628877 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0d14b0 con 0x7fbf9c0b5aa0 2024-03-23T07:57:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.665+0000 7fbf9a800640 1 --2- 172.21.15.105:0/1284628877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c0b5f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d90 tx=0x7fbf9402bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc288dc0644735c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbf99400640 1 -- 172.21.15.105:0/1284628877 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c001670 con 0x7fbf9c0b5aa0 2024-03-23T07:57:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbf99400640 1 -- 172.21.15.105:0/1284628877 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9402c8a0 con 0x7fbf9c0b5aa0 2024-03-23T07:57:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbf99400640 1 -- 172.21.15.105:0/1284628877 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9402cc10 con 0x7fbf9c0b5aa0 2024-03-23T07:57:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbfa1000640 1 -- 172.21.15.105:0/1284628877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b5aa0 msgr2=0x7fbf9c0b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbfa1000640 1 --2- 172.21.15.105:0/1284628877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c0b5f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d90 tx=0x7fbf9402bd70 comp rx=0 tx=0).stop 2024-03-23T07:57:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbfa1000640 1 -- 172.21.15.105:0/1284628877 shutdown_connections 2024-03-23T07:57:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbfa1000640 1 --2- 172.21.15.105:0/1284628877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c0b5f30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbfa1000640 1 --2- 172.21.15.105:0/1284628877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0b84b0 0x7fbf9c0d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.666+0000 7fbfa1000640 1 -- 172.21.15.105:0/1284628877 >> 172.21.15.105:0/1284628877 conn(0x7fbf9c0cf5d0 msgr2=0x7fbf9c0cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.667+0000 7fbfa1000640 1 -- 172.21.15.105:0/1284628877 shutdown_connections 2024-03-23T07:57:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.667+0000 7fbfa1000640 1 -- 172.21.15.105:0/1284628877 wait complete. 2024-03-23T07:57:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.668+0000 7fbfa1000640 1 Processor -- start 2024-03-23T07:57:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.668+0000 7fbfa1000640 1 -- start start 2024-03-23T07:57:18.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.669+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.669+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b84b0 0x7fbf9c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.669+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0cd740 0x7fbf9c0b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.669+0000 7fbfa1000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1559f0 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.669+0000 7fbfa1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c155cd0 con 0x7fbf9c0b5aa0 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.669+0000 7fbfa1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c155fb0 con 0x7fbf9c0cd740 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b84b0 0x7fbf9c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b84b0 0x7fbf9c15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36114/0 (socket says 172.21.15.105:36114) 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 -- 172.21.15.105:0/4236978316 learned_addr learned my addr 172.21.15.105:0/4236978316 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf99e00640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 -- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0cd740 msgr2=0x7fbf9c0b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0cd740 0x7fbf9c0b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 -- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf9c0b5aa0 msgr2=0x7fbf9c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c15a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 -- 172.21.15.105:0/4236978316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009a70 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf99e00640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:18.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf93e00640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b84b0 0x7fbf9c15a870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400d800 tx=0x7fbf8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:18.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf92000640 1 -- 172.21.15.105:0/4236978316 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400b6c0 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.670+0000 7fbf92000640 1 -- 172.21.15.105:0/4236978316 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8400b820 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.671+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c156290 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.671+0000 7fbf92000640 1 -- 172.21.15.105:0/4236978316 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400bbb0 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.671+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c14c860 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.672+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c0b5680 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.673+0000 7fbf92000640 1 -- 172.21.15.105:0/4236978316 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbf8400bd10 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.673+0000 7fbf92000640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf7806ac00 0x7fbf7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:18.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.674+0000 7fbf92000640 1 -- 172.21.15.105:0/4236978316 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbf84089c50 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.674+0000 7fbf99e00640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf7806ac00 0x7fbf7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:18.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.674+0000 7fbf99e00640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf7806ac00 0x7fbf7806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c007a00 tx=0x7fbf8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:18.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.678+0000 7fbf92000640 1 -- 172.21.15.105:0/4236978316 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbf8408f020 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.871+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf9c073390 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.872+0000 7fbf92000640 1 -- 172.21.15.105:0/4236978316 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf84054ae0 con 0x7fbf9c0b84b0 2024-03-23T07:57:18.872 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf7806ac00 msgr2=0x7fbf7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf7806ac00 0x7fbf7806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c007a00 tx=0x7fbf8c009040 comp rx=0 tx=0).stop 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b84b0 msgr2=0x7fbf9c15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b84b0 0x7fbf9c15a870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400d800 tx=0x7fbf8400dcc0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 shutdown_connections 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbf7806ac00 0x7fbf7806d0c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf9c0cd740 0x7fbf9c0b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf9c0b84b0 0x7fbf9c15a870 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 --2- 172.21.15.105:0/4236978316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf9c0b5aa0 0x7fbf9c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:18.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 >> 172.21.15.105:0/4236978316 conn(0x7fbf9c0cf5d0 msgr2=0x7fbf9c15f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:18.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 shutdown_connections 2024-03-23T07:57:18.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:18.875+0000 7fbfa1000640 1 -- 172.21.15.105:0/4236978316 wait complete. 2024-03-23T07:57:18.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:19.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.107+0000 7f430fa00640 1 Processor -- start 2024-03-23T07:57:19.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.107+0000 7f430fa00640 1 -- start start 2024-03-23T07:57:19.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f430fa00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43080ce930 0x7f43080add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f430fa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f430806ce90 0x7f430806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f430fa00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43080cdba0 con 0x7f43080cd3e0 2024-03-23T07:57:19.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f430fa00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430806c760 con 0x7f43080ce930 2024-03-23T07:57:19.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f430fa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43080b42d0 con 0x7f430806ce90 2024-03-23T07:57:19.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f430d000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43080ce930 0x7f43080add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f4307e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f43080cd3e0 0x7f43080ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47764/0 (socket says 172.21.15.105:47764) 2024-03-23T07:57:19.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.109+0000 7f4307e00640 1 -- 172.21.15.105:0/308342076 learned_addr learned my addr 172.21.15.105:0/308342076 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:19.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.110+0000 7f430da00640 1 --2- 172.21.15.105:0/308342076 >> v2:172.21.15.132:3300/0 conn(0x7f430806ce90 0x7f430806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.110+0000 7f430d000640 1 -- 172.21.15.105:0/308342076 >> v2:172.21.15.132:3300/0 conn(0x7f430806ce90 msgr2=0x7f430806d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.110+0000 7f430d000640 1 --2- 172.21.15.105:0/308342076 >> v2:172.21.15.132:3300/0 conn(0x7f430806ce90 0x7f430806d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.110+0000 7f430d000640 1 -- 172.21.15.105:0/308342076 >> v1:172.21.15.105:6789/0 conn(0x7f43080cd3e0 legacy=0x7f43080ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.110+0000 7f430d000640 1 -- 172.21.15.105:0/308342076 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43080cde80 con 0x7f43080ce930 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.110+0000 7f4307400640 1 -- 172.21.15.105:0/308342076 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115506372 0 0) 0x7f43080cdba0 con 0x7f43080cd3e0 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.111+0000 7f430d000640 1 --2- 172.21.15.105:0/308342076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080add30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f430806c4b0 tx=0x7f42fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b2cd4606bde2e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.111+0000 7f4307400640 1 -- 172.21.15.105:0/308342076 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc031830 con 0x7f43080ce930 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.111+0000 7f4307400640 1 -- 172.21.15.105:0/308342076 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc031990 con 0x7f43080ce930 2024-03-23T07:57:19.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.111+0000 7f4307400640 1 -- 172.21.15.105:0/308342076 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc031d20 con 0x7f43080ce930 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 -- 172.21.15.105:0/308342076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 msgr2=0x7f43080add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 --2- 172.21.15.105:0/308342076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080add30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f430806c4b0 tx=0x7f42fc02b8d0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 -- 172.21.15.105:0/308342076 shutdown_connections 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 --2- 172.21.15.105:0/308342076 >> v2:172.21.15.132:3300/0 conn(0x7f430806ce90 0x7f430806d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 --2- 172.21.15.105:0/308342076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080add30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 -- 172.21.15.105:0/308342076 >> 172.21.15.105:0/308342076 conn(0x7f430806a0b0 msgr2=0x7f430806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 -- 172.21.15.105:0/308342076 shutdown_connections 2024-03-23T07:57:19.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.112+0000 7f430fa00640 1 -- 172.21.15.105:0/308342076 wait complete. 2024-03-23T07:57:19.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.113+0000 7f430fa00640 1 Processor -- start 2024-03-23T07:57:19.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.114+0000 7f430fa00640 1 -- start start 2024-03-23T07:57:19.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.114+0000 7f430fa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f430806ce90 0x7f43080ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.114+0000 7f430fa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080cd3e0 0x7f4308150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.114+0000 7f430fa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.114+0000 7f430fa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43080aa110 con 0x7f43080ce930 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.114+0000 7f430fa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43080a9590 con 0x7f43080cd3e0 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.114+0000 7f430fa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43080a9870 con 0x7f430806ce90 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080a8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36130/0 (socket says 172.21.15.105:36130) 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 -- 172.21.15.105:0/1989987414 learned_addr learned my addr 172.21.15.105:0/1989987414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f4307e00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080cd3e0 0x7f4308150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430d000640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f430806ce90 0x7f43080ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 -- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f430806ce90 msgr2=0x7f43080ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f430806ce90 0x7f43080ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 -- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080cd3e0 msgr2=0x7f4308150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080cd3e0 0x7f4308150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430da00640 1 -- 172.21.15.105:0/1989987414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc009400 con 0x7f43080ce930 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f4307e00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080cd3e0 0x7f4308150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.115+0000 7f430d000640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f430806ce90 0x7f43080ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.116+0000 7f430da00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080a8cf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f430000cd70 tx=0x7f430000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.116+0000 7f4304c00640 1 -- 172.21.15.105:0/1989987414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300003b90 con 0x7f43080ce930 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.116+0000 7f4304c00640 1 -- 172.21.15.105:0/1989987414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300003cf0 con 0x7f43080ce930 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.116+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43080a9b50 con 0x7f43080ce930 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.116+0000 7f4304c00640 1 -- 172.21.15.105:0/1989987414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300004240 con 0x7f43080ce930 2024-03-23T07:57:19.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.116+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4308150690 con 0x7f43080ce930 2024-03-23T07:57:19.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.117+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43080c35c0 con 0x7f43080ce930 2024-03-23T07:57:19.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.118+0000 7f4304c00640 1 -- 172.21.15.105:0/1989987414 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f43000043a0 con 0x7f43080ce930 2024-03-23T07:57:19.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.119+0000 7f4304c00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f42e806ac00 0x7f42e806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.119+0000 7f4304c00640 1 -- 172.21.15.105:0/1989987414 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4300012070 con 0x7f43080ce930 2024-03-23T07:57:19.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.119+0000 7f430d000640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f42e806ac00 0x7f42e806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.120+0000 7f430d000640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f42e806ac00 0x7f42e806d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f42fc036000 tx=0x7f42fc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.123+0000 7f4304c00640 1 -- 172.21.15.105:0/1989987414 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f43080c35c0 con 0x7f43080ce930 2024-03-23T07:57:19.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.317+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f43080c35c0 con 0x7f43080ce930 2024-03-23T07:57:19.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.317+0000 7f4304c00640 1 -- 172.21.15.105:0/1989987414 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f43080c35c0 con 0x7f43080ce930 2024-03-23T07:57:19.317 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:19.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f42e806ac00 msgr2=0x7f42e806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f42e806ac00 0x7f42e806d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f42fc036000 tx=0x7f42fc03c040 comp rx=0 tx=0).stop 2024-03-23T07:57:19.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 msgr2=0x7f43080a8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080a8cf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f430000cd70 tx=0x7f430000a660 comp rx=0 tx=0).stop 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 shutdown_connections 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43080ce930 0x7f43080a8cf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080cd3e0 0x7f4308150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f42e806ac00 0x7f42e806d0c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 --2- 172.21.15.105:0/1989987414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f430806ce90 0x7f43080ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 >> 172.21.15.105:0/1989987414 conn(0x7f430806a0b0 msgr2=0x7f43080b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 shutdown_connections 2024-03-23T07:57:19.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.320+0000 7f430fa00640 1 -- 172.21.15.105:0/1989987414 wait complete. 2024-03-23T07:57:19.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.549+0000 7f1d5ea00640 1 Processor -- start 2024-03-23T07:57:19.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.549+0000 7f1d5ea00640 1 -- start start 2024-03-23T07:57:19.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.550+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5815b560 0x7f1d5815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.550+0000 7f1d5ea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d580b39b0 0x7f1d580b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d5ea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5806c6f0 con 0x7f1d5815e500 2024-03-23T07:57:19.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d5ea00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d580b4370 con 0x7f1d580b39b0 2024-03-23T07:57:19.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d5ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5806c9d0 con 0x7f1d5815b560 2024-03-23T07:57:19.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d57400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d580b39b0 0x7f1d580b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d57400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d580b39b0 0x7f1d580b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36134/0 (socket says 172.21.15.105:36134) 2024-03-23T07:57:19.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d57400640 1 -- 172.21.15.105:0/2205000068 learned_addr learned my addr 172.21.15.105:0/2205000068 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:19.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d57400640 1 -- 172.21.15.105:0/2205000068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5815b560 msgr2=0x7f1d5815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:19.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d57400640 1 --2- 172.21.15.105:0/2205000068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5815b560 0x7f1d5815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d57400640 1 -- 172.21.15.105:0/2205000068 >> v1:172.21.15.132:6789/0 conn(0x7f1d5815e500 legacy=0x7f1d5815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:19.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.551+0000 7f1d57400640 1 -- 172.21.15.105:0/2205000068 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5806ccb0 con 0x7f1d580b39b0 2024-03-23T07:57:19.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.552+0000 7f1d57400640 1 --2- 172.21.15.105:0/2205000068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d580b39b0 0x7f1d580b3e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009870 tx=0x7f1d4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb49a47eb2a3135 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.553+0000 7f1d56a00640 1 -- 172.21.15.105:0/2205000068 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44031640 con 0x7f1d580b39b0 2024-03-23T07:57:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.553+0000 7f1d56a00640 1 -- 172.21.15.105:0/2205000068 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d440317a0 con 0x7f1d580b39b0 2024-03-23T07:57:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.553+0000 7f1d56a00640 1 -- 172.21.15.105:0/2205000068 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44031b30 con 0x7f1d580b39b0 2024-03-23T07:57:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.553+0000 7f1d5ea00640 1 -- 172.21.15.105:0/2205000068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d580b39b0 msgr2=0x7f1d580b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.553+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/2205000068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d580b39b0 0x7f1d580b3e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009870 tx=0x7f1d4402b8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.554+0000 7f1d5ea00640 1 -- 172.21.15.105:0/2205000068 shutdown_connections 2024-03-23T07:57:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.554+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/2205000068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d580b39b0 0x7f1d580b3e30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.554+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/2205000068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5815b560 0x7f1d5815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.554+0000 7f1d5ea00640 1 -- 172.21.15.105:0/2205000068 >> 172.21.15.105:0/2205000068 conn(0x7f1d5806a3d0 msgr2=0x7f1d580b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.554+0000 7f1d5ea00640 1 -- 172.21.15.105:0/2205000068 shutdown_connections 2024-03-23T07:57:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.554+0000 7f1d5ea00640 1 -- 172.21.15.105:0/2205000068 wait complete. 2024-03-23T07:57:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.555+0000 7f1d5ea00640 1 Processor -- start 2024-03-23T07:57:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.556+0000 7f1d5ea00640 1 -- start start 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d580b39b0 0x7f1d580cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d5815b560 0x7f1d580cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5815e500 0x7f1d58151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d5ea00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d580cc5c0 con 0x7f1d5815b560 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d5ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d580cc8a0 con 0x7f1d5815e500 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d5ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d580ccb80 con 0x7f1d580b39b0 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d5815b560 0x7f1d580cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d5815b560 0x7f1d580cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36142/0 (socket says 172.21.15.105:36142) 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 -- 172.21.15.105:0/1128291893 learned_addr learned my addr 172.21.15.105:0/1128291893 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57e00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d580b39b0 0x7f1d580cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 -- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d580b39b0 msgr2=0x7f1d580cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d580b39b0 0x7f1d580cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 -- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5815e500 msgr2=0x7f1d58151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d5ca00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5815e500 0x7f1d58151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5815e500 0x7f1d58151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57400640 1 -- 172.21.15.105:0/1128291893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d440094d0 con 0x7f1d5815b560 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.557+0000 7f1d57e00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d580b39b0 0x7f1d580cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.558+0000 7f1d57400640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d5815b560 0x7f1d580cbfd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009870 tx=0x7f1d440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.558+0000 7f1d4fe00640 1 -- 172.21.15.105:0/1128291893 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4403ac60 con 0x7f1d5815b560 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.558+0000 7f1d4fe00640 1 -- 172.21.15.105:0/1128291893 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4403adc0 con 0x7f1d5815b560 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.558+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d581f2790 con 0x7f1d5815b560 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.558+0000 7f1d4fe00640 1 -- 172.21.15.105:0/1128291893 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44044630 con 0x7f1d5815b560 2024-03-23T07:57:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.558+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d580cd520 con 0x7f1d5815b560 2024-03-23T07:57:19.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.559+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d581f2cb0 con 0x7f1d5815b560 2024-03-23T07:57:19.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.560+0000 7f1d4fe00640 1 -- 172.21.15.105:0/1128291893 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1d4402f050 con 0x7f1d5815b560 2024-03-23T07:57:19.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.561+0000 7f1d4fe00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d3806ac00 0x7f1d3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.561+0000 7f1d4fe00640 1 -- 172.21.15.105:0/1128291893 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1d440ab6c0 con 0x7f1d5815b560 2024-03-23T07:57:19.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.561+0000 7f1d57e00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d3806ac00 0x7f1d3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.562+0000 7f1d57e00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d3806ac00 0x7f1d3806d0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d48006fd0 tx=0x7f1d48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.566+0000 7f1d4fe00640 1 -- 172.21.15.105:0/1128291893 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1d581f2cb0 con 0x7f1d5815b560 2024-03-23T07:57:19.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.745+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d581f2cb0 con 0x7f1d5815b560 2024-03-23T07:57:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.747+0000 7f1d4fe00640 1 -- 172.21.15.105:0/1128291893 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f1d581f2cb0 con 0x7f1d5815b560 2024-03-23T07:57:19.747 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:19.748 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d3806ac00 msgr2=0x7f1d3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d3806ac00 0x7f1d3806d0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d48006fd0 tx=0x7f1d48008040 comp rx=0 tx=0).stop 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d5815b560 msgr2=0x7f1d580cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d5815b560 0x7f1d580cbfd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009870 tx=0x7f1d440046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 shutdown_connections 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5815e500 0x7f1d58151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d5815b560 0x7f1d580cbfd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1d3806ac00 0x7f1d3806d0c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 --2- 172.21.15.105:0/1128291893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d580b39b0 0x7f1d580cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 >> 172.21.15.105:0/1128291893 conn(0x7f1d5806a3d0 msgr2=0x7f1d580b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 shutdown_connections 2024-03-23T07:57:19.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.750+0000 7f1d5ea00640 1 -- 172.21.15.105:0/1128291893 wait complete. 2024-03-23T07:57:19.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:19.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:19.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.977+0000 7f3dc9e00640 1 Processor -- start 2024-03-23T07:57:19.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.977+0000 7f3dc9e00640 1 -- start start 2024-03-23T07:57:19.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.978+0000 7f3dc9e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3dc415a850 0x7f3dc40cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.978+0000 7f3dc9e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3dc40cacf0 0x7f3dc40cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.978+0000 7f3dc9e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc415e320 con 0x7f3dc40b22b0 2024-03-23T07:57:19.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.978+0000 7f3dc9e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc40cb6c0 con 0x7f3dc40cacf0 2024-03-23T07:57:19.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.978+0000 7f3dc9e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc415ac30 con 0x7f3dc415a850 2024-03-23T07:57:19.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dc9400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3dc40cacf0 0x7f3dc40cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dbbe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3dc40b22b0 0x7f3dc415eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47776/0 (socket says 172.21.15.105:47776) 2024-03-23T07:57:19.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dbbe00640 1 -- 172.21.15.105:0/754308906 learned_addr learned my addr 172.21.15.105:0/754308906 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:19.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dc8a00640 1 --2- 172.21.15.105:0/754308906 >> v2:172.21.15.132:3300/0 conn(0x7f3dc415a850 0x7f3dc40cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dc3e00640 1 -- 172.21.15.105:0/754308906 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549408407 0 0) 0x7f3dc415e320 con 0x7f3dc40b22b0 2024-03-23T07:57:19.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dc9400640 1 -- 172.21.15.105:0/754308906 >> v2:172.21.15.132:3300/0 conn(0x7f3dc415a850 msgr2=0x7f3dc40cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:19.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dc9400640 1 --2- 172.21.15.105:0/754308906 >> v2:172.21.15.132:3300/0 conn(0x7f3dc415a850 0x7f3dc40cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dc9400640 1 -- 172.21.15.105:0/754308906 >> v1:172.21.15.105:6789/0 conn(0x7f3dc40b22b0 legacy=0x7f3dc415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.979+0000 7f3dc9400640 1 -- 172.21.15.105:0/754308906 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40cda20 con 0x7f3dc40cacf0 2024-03-23T07:57:19.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.980+0000 7f3dc9400640 1 --2- 172.21.15.105:0/754308906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40cacf0 0x7f3dc40cb180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3dc406adb0 tx=0x7f3dbc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ee89af2f6a0e7d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc3e00640 1 -- 172.21.15.105:0/754308906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc02cc80 con 0x7f3dc40cacf0 2024-03-23T07:57:19.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc3e00640 1 -- 172.21.15.105:0/754308906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc02cde0 con 0x7f3dc40cacf0 2024-03-23T07:57:19.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc3e00640 1 -- 172.21.15.105:0/754308906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc034660 con 0x7f3dc40cacf0 2024-03-23T07:57:19.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 -- 172.21.15.105:0/754308906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40cacf0 msgr2=0x7f3dc40cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/754308906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40cacf0 0x7f3dc40cb180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3dc406adb0 tx=0x7f3dbc02bed0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 -- 172.21.15.105:0/754308906 shutdown_connections 2024-03-23T07:57:19.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/754308906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40cacf0 0x7f3dc40cb180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/754308906 >> v2:172.21.15.132:3300/0 conn(0x7f3dc415a850 0x7f3dc40cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 -- 172.21.15.105:0/754308906 >> 172.21.15.105:0/754308906 conn(0x7f3dc40b01b0 msgr2=0x7f3dc40b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:19.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 -- 172.21.15.105:0/754308906 shutdown_connections 2024-03-23T07:57:19.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.981+0000 7f3dc9e00640 1 -- 172.21.15.105:0/754308906 wait complete. 2024-03-23T07:57:19.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.984+0000 7f3dc9e00640 1 Processor -- start 2024-03-23T07:57:19.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.984+0000 7f3dc9e00640 1 -- start start 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40b22b0 0x7f3dc409bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc40cacf0 0x7f3dc415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc415a850 0x7f3dc41593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc9e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc41f1fa0 con 0x7f3dc40b22b0 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc9e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc415c0d0 con 0x7f3dc40cacf0 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc9e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc415c3b0 con 0x7f3dc415a850 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40b22b0 0x7f3dc409bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40b22b0 0x7f3dc409bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36166/0 (socket says 172.21.15.105:36166) 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.985+0000 7f3dc8a00640 1 -- 172.21.15.105:0/1216629686 learned_addr learned my addr 172.21.15.105:0/1216629686 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc8a00640 1 -- 172.21.15.105:0/1216629686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc415a850 msgr2=0x7f3dc41593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc8a00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc415a850 0x7f3dc41593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc8a00640 1 -- 172.21.15.105:0/1216629686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc40cacf0 msgr2=0x7f3dc415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc8a00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc40cacf0 0x7f3dc415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc8a00640 1 -- 172.21.15.105:0/1216629686 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009a70 con 0x7f3dc40b22b0 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc8a00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40b22b0 0x7f3dc409bcd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3db400d800 tx=0x7f3db400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc1600640 1 -- 172.21.15.105:0/1216629686 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db400b6c0 con 0x7f3dc40b22b0 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc1600640 1 -- 172.21.15.105:0/1216629686 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db400b820 con 0x7f3dc40b22b0 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc415c690 con 0x7f3dc40b22b0 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc1600640 1 -- 172.21.15.105:0/1216629686 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db400bbb0 con 0x7f3dc40b22b0 2024-03-23T07:57:19.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.986+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc409c210 con 0x7f3dc40b22b0 2024-03-23T07:57:19.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.988+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc404e030 con 0x7f3dc40b22b0 2024-03-23T07:57:19.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.988+0000 7f3dc1600640 1 -- 172.21.15.105:0/1216629686 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3db400bd10 con 0x7f3dc40b22b0 2024-03-23T07:57:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.989+0000 7f3dc1600640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3da006ac30 0x7f3da006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.990+0000 7f3dc1600640 1 -- 172.21.15.105:0/1216629686 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3db4089ca0 con 0x7f3dc40b22b0 2024-03-23T07:57:19.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.990+0000 7f3dbbe00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3da006ac30 0x7f3da006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:19.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.990+0000 7f3dbbe00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3da006ac30 0x7f3da006d0f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3dac007fd0 tx=0x7f3dac007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:19.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:19.994+0000 7f3dc1600640 1 -- 172.21.15.105:0/1216629686 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3db4054a90 con 0x7f3dc40b22b0 2024-03-23T07:57:20.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.188+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3dc40ccaa0 con 0x7f3dc40b22b0 2024-03-23T07:57:20.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.189+0000 7f3dc1600640 1 -- 172.21.15.105:0/1216629686 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3db40580b0 con 0x7f3dc40b22b0 2024-03-23T07:57:20.189 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:20.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3da006ac30 msgr2=0x7f3da006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3da006ac30 0x7f3da006d0f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3dac007fd0 tx=0x7f3dac007960 comp rx=0 tx=0).stop 2024-03-23T07:57:20.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40b22b0 msgr2=0x7f3dc409bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40b22b0 0x7f3dc409bcd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3db400d800 tx=0x7f3db400dcc0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 shutdown_connections 2024-03-23T07:57:20.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3da006ac30 0x7f3da006d0f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc415a850 0x7f3dc41593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc40cacf0 0x7f3dc415b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 --2- 172.21.15.105:0/1216629686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dc40b22b0 0x7f3dc409bcd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 >> 172.21.15.105:0/1216629686 conn(0x7f3dc40b01b0 msgr2=0x7f3dc40b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:20.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 shutdown_connections 2024-03-23T07:57:20.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.192+0000 7f3dc9e00640 1 -- 172.21.15.105:0/1216629686 wait complete. 2024-03-23T07:57:20.209 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:20.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.414+0000 7f36aa600640 1 Processor -- start 2024-03-23T07:57:20.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.414+0000 7f36aa600640 1 -- start start 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36aa600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f36a4159cd0 0x7f36a40ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36aa600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36a40cc200 0x7f36a40cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36aa600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a40c8c90 con 0x7f36a40ad7a0 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36aa600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a40cc6c0 con 0x7f36a40cc200 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36aa600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a40cc9a0 con 0x7f36a4159cd0 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36a40cc200 0x7f36a40cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36a40cc200 0x7f36a40cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36182/0 (socket says 172.21.15.105:36182) 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a3e00640 1 -- 172.21.15.105:0/1988604288 learned_addr learned my addr 172.21.15.105:0/1988604288 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a9200640 1 --2- 172.21.15.105:0/1988604288 >> v2:172.21.15.132:3300/0 conn(0x7f36a4159cd0 0x7f36a40ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a3e00640 1 -- 172.21.15.105:0/1988604288 >> v2:172.21.15.132:3300/0 conn(0x7f36a4159cd0 msgr2=0x7f36a40ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:20.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a3e00640 1 --2- 172.21.15.105:0/1988604288 >> v2:172.21.15.132:3300/0 conn(0x7f36a4159cd0 0x7f36a40ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a3e00640 1 -- 172.21.15.105:0/1988604288 >> v1:172.21.15.105:6789/0 conn(0x7f36a40ad7a0 legacy=0x7f36a415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.416+0000 7f36a3e00640 1 -- 172.21.15.105:0/1988604288 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40ccc80 con 0x7f36a40cc200 2024-03-23T07:57:20.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.417+0000 7f36a9200640 1 --2- 172.21.15.105:0/1988604288 >> v2:172.21.15.132:3300/0 conn(0x7f36a4159cd0 0x7f36a40ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:20.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.418+0000 7f36a3e00640 1 --2- 172.21.15.105:0/1988604288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40cc200 0x7f36a40cab30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f36a406e4f0 tx=0x7f369002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d62785d0226ed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:20.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.418+0000 7f36a3400640 1 -- 172.21.15.105:0/1988604288 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690031870 con 0x7f36a40cc200 2024-03-23T07:57:20.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.418+0000 7f36a3400640 1 -- 172.21.15.105:0/1988604288 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36900319d0 con 0x7f36a40cc200 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.418+0000 7f36a3400640 1 -- 172.21.15.105:0/1988604288 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690031d60 con 0x7f36a40cc200 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 -- 172.21.15.105:0/1988604288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40cc200 msgr2=0x7f36a40cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 --2- 172.21.15.105:0/1988604288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40cc200 0x7f36a40cab30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f36a406e4f0 tx=0x7f369002b910 comp rx=0 tx=0).stop 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 -- 172.21.15.105:0/1988604288 shutdown_connections 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 --2- 172.21.15.105:0/1988604288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40cc200 0x7f36a40cab30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 --2- 172.21.15.105:0/1988604288 >> v2:172.21.15.132:3300/0 conn(0x7f36a4159cd0 0x7f36a40ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 -- 172.21.15.105:0/1988604288 >> 172.21.15.105:0/1988604288 conn(0x7f36a40b01b0 msgr2=0x7f36a40b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 -- 172.21.15.105:0/1988604288 shutdown_connections 2024-03-23T07:57:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.419+0000 7f36aa600640 1 -- 172.21.15.105:0/1988604288 wait complete. 2024-03-23T07:57:20.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.421+0000 7f36aa600640 1 Processor -- start 2024-03-23T07:57:20.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.421+0000 7f36aa600640 1 -- start start 2024-03-23T07:57:20.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36aa600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40ad7a0 0x7f36a4097230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36aa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a40cc200 0x7f36a4151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36aa600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a4159cd0 0x7f36a406c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36aa600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40707f0 con 0x7f36a40ad7a0 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36aa600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a406fee0 con 0x7f36a4159cd0 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36aa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a406cde0 con 0x7f36a40cc200 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40ad7a0 0x7f36a4097230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40ad7a0 0x7f36a4097230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36198/0 (socket says 172.21.15.105:36198) 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 -- 172.21.15.105:0/518518538 learned_addr learned my addr 172.21.15.105:0/518518538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 -- 172.21.15.105:0/518518538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a40cc200 msgr2=0x7f36a4151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a40cc200 0x7f36a4151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 -- 172.21.15.105:0/518518538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a4159cd0 msgr2=0x7f36a406c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a4159cd0 0x7f36a406c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.422+0000 7f36a9200640 1 -- 172.21.15.105:0/518518538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36900094d0 con 0x7f36a40ad7a0 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.423+0000 7f36a9200640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40ad7a0 0x7f36a4097230 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f369800c820 tx=0x7f369800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.423+0000 7f36a0c00640 1 -- 172.21.15.105:0/518518538 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698007a30 con 0x7f36a40ad7a0 2024-03-23T07:57:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.423+0000 7f36a0c00640 1 -- 172.21.15.105:0/518518538 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3698007b90 con 0x7f36a40ad7a0 2024-03-23T07:57:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.423+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a4166870 con 0x7f36a40ad7a0 2024-03-23T07:57:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.423+0000 7f36a0c00640 1 -- 172.21.15.105:0/518518538 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800ce10 con 0x7f36a40ad7a0 2024-03-23T07:57:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.423+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a4070410 con 0x7f36a40ad7a0 2024-03-23T07:57:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.424+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a404e030 con 0x7f36a40ad7a0 2024-03-23T07:57:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.425+0000 7f36a0c00640 1 -- 172.21.15.105:0/518518538 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f36980113f0 con 0x7f36a40ad7a0 2024-03-23T07:57:20.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.426+0000 7f36a0c00640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f368406ac00 0x7f368406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.426+0000 7f36a0c00640 1 -- 172.21.15.105:0/518518538 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f36980897b0 con 0x7f36a40ad7a0 2024-03-23T07:57:20.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.426+0000 7f36a3e00640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f368406ac00 0x7f368406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.427+0000 7f36a3e00640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f368406ac00 0x7f368406d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3690036000 tx=0x7f3690030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:20.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.430+0000 7f36a0c00640 1 -- 172.21.15.105:0/518518538 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3698054630 con 0x7f36a40ad7a0 2024-03-23T07:57:20.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.618+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36a40c9d30 con 0x7f36a40ad7a0 2024-03-23T07:57:20.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.619+0000 7f36a0c00640 1 -- 172.21.15.105:0/518518538 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3698057c50 con 0x7f36a40ad7a0 2024-03-23T07:57:20.618 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:20.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f368406ac00 msgr2=0x7f368406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f368406ac00 0x7f368406d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3690036000 tx=0x7f3690030000 comp rx=0 tx=0).stop 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40ad7a0 msgr2=0x7f36a4097230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40ad7a0 0x7f36a4097230 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f369800c820 tx=0x7f369800cce0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 shutdown_connections 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a4159cd0 0x7f36a406c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f368406ac00 0x7f368406d0c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a40cc200 0x7f36a4151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 --2- 172.21.15.105:0/518518538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a40ad7a0 0x7f36a4097230 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.621+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 >> 172.21.15.105:0/518518538 conn(0x7f36a40b01b0 msgr2=0x7f36a40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.622+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 shutdown_connections 2024-03-23T07:57:20.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.622+0000 7f36aa600640 1 -- 172.21.15.105:0/518518538 wait complete. 2024-03-23T07:57:20.638 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:20.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.843+0000 7f2f71c00640 1 Processor -- start 2024-03-23T07:57:20.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.844+0000 7f2f71c00640 1 -- start start 2024-03-23T07:57:20.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f71c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2f6c0b84b0 0x7f2f6c0d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f71c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f6c0b5aa0 0x7f2f6c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f71c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0d1280 con 0x7f2f6c0cd740 2024-03-23T07:57:20.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f71c00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0d20c0 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f71c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0b8940 con 0x7f2f6c0b84b0 2024-03-23T07:57:20.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f6aa00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2f6c0cd740 0x7f2f6c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47786/0 (socket says 172.21.15.105:47786) 2024-03-23T07:57:20.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f6aa00640 1 -- 172.21.15.105:0/1574343879 learned_addr learned my addr 172.21.15.105:0/1574343879 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:20.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f6b400640 1 --2- 172.21.15.105:0/1574343879 >> v2:172.21.15.105:3300/0 conn(0x7f2f6c0b5aa0 0x7f2f6c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.845+0000 7f2f6a000640 1 --2- 172.21.15.105:0/1574343879 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c0b84b0 0x7f2f6c0d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.846+0000 7f2f69600640 1 -- 172.21.15.105:0/1574343879 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521364927 0 0) 0x7f2f6c0d1280 con 0x7f2f6c0cd740 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.846+0000 7f2f69600640 1 -- 172.21.15.105:0/1574343879 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c003520 con 0x7f2f6c0cd740 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.846+0000 7f2f6b400640 1 -- 172.21.15.105:0/1574343879 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c0b84b0 msgr2=0x7f2f6c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.846+0000 7f2f6b400640 1 --2- 172.21.15.105:0/1574343879 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c0b84b0 0x7f2f6c0d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.846+0000 7f2f6b400640 1 -- 172.21.15.105:0/1574343879 >> v1:172.21.15.105:6789/0 conn(0x7f2f6c0cd740 legacy=0x7f2f6c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.846+0000 7f2f6b400640 1 -- 172.21.15.105:0/1574343879 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c0d14b0 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.846+0000 7f2f6a000640 1 --2- 172.21.15.105:0/1574343879 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c0b84b0 0x7f2f6c0d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:20.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.847+0000 7f2f6b400640 1 --2- 172.21.15.105:0/1574343879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c0b5f30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0b8050 tx=0x7f2f6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8b6d90cf62509a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.847+0000 7f2f69600640 1 -- 172.21.15.105:0/1574343879 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64031830 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.847+0000 7f2f69600640 1 -- 172.21.15.105:0/1574343879 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64031990 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.847+0000 7f2f69600640 1 -- 172.21.15.105:0/1574343879 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64031d20 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 -- 172.21.15.105:0/1574343879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 msgr2=0x7f2f6c0b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 --2- 172.21.15.105:0/1574343879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c0b5f30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0b8050 tx=0x7f2f6402b8d0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 -- 172.21.15.105:0/1574343879 shutdown_connections 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 --2- 172.21.15.105:0/1574343879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c0b5f30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 --2- 172.21.15.105:0/1574343879 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c0b84b0 0x7f2f6c0d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 -- 172.21.15.105:0/1574343879 >> 172.21.15.105:0/1574343879 conn(0x7f2f6c0cf5d0 msgr2=0x7f2f6c0cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 -- 172.21.15.105:0/1574343879 shutdown_connections 2024-03-23T07:57:20.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.848+0000 7f2f71c00640 1 -- 172.21.15.105:0/1574343879 wait complete. 2024-03-23T07:57:20.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.850+0000 7f2f71c00640 1 Processor -- start 2024-03-23T07:57:20.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.850+0000 7f2f71c00640 1 -- start start 2024-03-23T07:57:20.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f71c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f71c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0b84b0 0x7f2f6c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f71c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c0cd740 0x7f2f6c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f71c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c9a40 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f71c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c15a410 con 0x7f2f6c0b84b0 2024-03-23T07:57:20.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f71c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c9c90 con 0x7f2f6c0cd740 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36222/0 (socket says 172.21.15.105:36222) 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f6aa00640 1 -- 172.21.15.105:0/342624133 learned_addr learned my addr 172.21.15.105:0/342624133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f6aa00640 1 -- 172.21.15.105:0/342624133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c0cd740 msgr2=0x7f2f6c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.851+0000 7f2f6aa00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c0cd740 0x7f2f6c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f6aa00640 1 -- 172.21.15.105:0/342624133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0b84b0 msgr2=0x7f2f6c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f6a000640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0b84b0 0x7f2f6c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f6aa00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0b84b0 0x7f2f6c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f6aa00640 1 -- 172.21.15.105:0/342624133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64009400 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f6aa00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c159ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c410 tx=0x7f2f5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:20.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f62a00640 1 -- 172.21.15.105:0/342624133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c005ce0 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f62a00640 1 -- 172.21.15.105:0/342624133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c005e40 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f62a00640 1 -- 172.21.15.105:0/342624133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0072f0 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c0c9f70 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.852+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c0ca530 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.853+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c073390 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.854+0000 7f2f62a00640 1 -- 172.21.15.105:0/342624133 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2f5c007510 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.855+0000 7f2f62a00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f4806ac00 0x7f2f4806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:20.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.855+0000 7f2f62a00640 1 -- 172.21.15.105:0/342624133 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2f5c088d40 con 0x7f2f6c0b5aa0 2024-03-23T07:57:20.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.856+0000 7f2f6a000640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f4806ac00 0x7f2f4806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:20.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.856+0000 7f2f6a000640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f4806ac00 0x7f2f4806d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0cde70 tx=0x7f2f540084d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:20.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:20.859+0000 7f2f62a00640 1 -- 172.21.15.105:0/342624133 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2f5c053c80 con 0x7f2f6c0b5aa0 2024-03-23T07:57:21.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.039+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f6c0b20b0 con 0x7f2f6c0b5aa0 2024-03-23T07:57:21.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.039+0000 7f2f62a00640 1 -- 172.21.15.105:0/342624133 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f5c0170a0 con 0x7f2f6c0b5aa0 2024-03-23T07:57:21.039 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:21.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f4806ac00 msgr2=0x7f2f4806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f4806ac00 0x7f2f4806d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0cde70 tx=0x7f2f540084d0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 msgr2=0x7f2f6c159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c159ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c410 tx=0x7f2f5c00c8d0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 shutdown_connections 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f4806ac00 0x7f2f4806d0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c0cd740 0x7f2f6c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0b84b0 0x7f2f6c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 --2- 172.21.15.105:0/342624133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f6c0b5aa0 0x7f2f6c159ed0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 >> 172.21.15.105:0/342624133 conn(0x7f2f6c0cf5d0 msgr2=0x7f2f6c0b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 shutdown_connections 2024-03-23T07:57:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.042+0000 7f2f71c00640 1 -- 172.21.15.105:0/342624133 wait complete. 2024-03-23T07:57:21.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:21.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.262+0000 7f18e3800640 1 Processor -- start 2024-03-23T07:57:21.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.262+0000 7f18e3800640 1 -- start start 2024-03-23T07:57:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.264+0000 7f18e3800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f18dc0ce930 0x7f18dc0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.264+0000 7f18e3800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc159c10 con 0x7f18dc159830 2024-03-23T07:57:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.264+0000 7f18e3800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc0cd840 con 0x7f18dc0ca7b0 2024-03-23T07:57:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.264+0000 7f18e3800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc06de50 con 0x7f18dc0ce930 2024-03-23T07:57:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.264+0000 7f18e0e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f18dc159830 0x7f18dc0cc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47788/0 (socket says 172.21.15.105:47788) 2024-03-23T07:57:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.264+0000 7f18e0e00640 1 -- 172.21.15.105:0/1329363110 learned_addr learned my addr 172.21.15.105:0/1329363110 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.264+0000 7f18e1800640 1 --2- 172.21.15.105:0/1329363110 >> v2:172.21.15.105:3300/0 conn(0x7f18dc0ce930 0x7f18dc0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.265+0000 7f18e1800640 1 -- 172.21.15.105:0/1329363110 >> v1:172.21.15.132:6789/0 conn(0x7f18dc0ca7b0 legacy=0x7f18dc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.265+0000 7f18e1800640 1 -- 172.21.15.105:0/1329363110 >> v1:172.21.15.105:6789/0 conn(0x7f18dc159830 legacy=0x7f18dc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.265+0000 7f18e1800640 1 -- 172.21.15.105:0/1329363110 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0ccc30 con 0x7f18dc0ce930 2024-03-23T07:57:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.266+0000 7f18e1800640 1 --2- 172.21.15.105:0/1329363110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc0ce930 0x7f18dc0cedc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009dc0 tx=0x7f18d402e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fb462bdb8d300e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.266+0000 7f18dbe00640 1 -- 172.21.15.105:0/1329363110 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4036440 con 0x7f18dc0ce930 2024-03-23T07:57:21.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.266+0000 7f18dbe00640 1 -- 172.21.15.105:0/1329363110 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d40365a0 con 0x7f18dc0ce930 2024-03-23T07:57:21.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.266+0000 7f18dbe00640 1 -- 172.21.15.105:0/1329363110 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4036930 con 0x7f18dc0ce930 2024-03-23T07:57:21.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.267+0000 7f18e3800640 1 -- 172.21.15.105:0/1329363110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc0ce930 msgr2=0x7f18dc0cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.267+0000 7f18e3800640 1 --2- 172.21.15.105:0/1329363110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc0ce930 0x7f18dc0cedc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009dc0 tx=0x7f18d402e140 comp rx=0 tx=0).stop 2024-03-23T07:57:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.267+0000 7f18e3800640 1 -- 172.21.15.105:0/1329363110 shutdown_connections 2024-03-23T07:57:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.267+0000 7f18e3800640 1 --2- 172.21.15.105:0/1329363110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc0ce930 0x7f18dc0cedc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.267+0000 7f18e3800640 1 -- 172.21.15.105:0/1329363110 >> 172.21.15.105:0/1329363110 conn(0x7f18dc06a0b0 msgr2=0x7f18dc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.267+0000 7f18e3800640 1 -- 172.21.15.105:0/1329363110 shutdown_connections 2024-03-23T07:57:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.267+0000 7f18e3800640 1 -- 172.21.15.105:0/1329363110 wait complete. 2024-03-23T07:57:21.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.268+0000 7f18e3800640 1 Processor -- start 2024-03-23T07:57:21.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.269+0000 7f18e3800640 1 -- start start 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.269+0000 7f18e3800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18dc0ca7b0 0x7f18dc1490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.269+0000 7f18e3800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18dc0ce930 0x7f18dc14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e3800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc159830 0x7f18dc150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e3800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0abbb0 con 0x7f18dc159830 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e3800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc150ac0 con 0x7f18dc0ce930 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e3800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc150da0 con 0x7f18dc0ca7b0 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc159830 0x7f18dc150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc159830 0x7f18dc150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36250/0 (socket says 172.21.15.105:36250) 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 -- 172.21.15.105:0/1099746702 learned_addr learned my addr 172.21.15.105:0/1099746702 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 -- 172.21.15.105:0/1099746702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18dc0ca7b0 msgr2=0x7f18dc1490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18dc0ca7b0 0x7f18dc1490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 -- 172.21.15.105:0/1099746702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18dc0ce930 msgr2=0x7f18dc14ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18dc0ce930 0x7f18dc14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 -- 172.21.15.105:0/1099746702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4009a70 con 0x7f18dc159830 2024-03-23T07:57:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.270+0000 7f18e1800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc159830 0x7f18dc150440 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f18d4039040 tx=0x7f18d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.271+0000 7f18d9600640 1 -- 172.21.15.105:0/1099746702 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d403f8f0 con 0x7f18dc159830 2024-03-23T07:57:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.271+0000 7f18d9600640 1 -- 172.21.15.105:0/1099746702 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d403fa50 con 0x7f18dc159830 2024-03-23T07:57:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.271+0000 7f18d9600640 1 -- 172.21.15.105:0/1099746702 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d403fde0 con 0x7f18dc159830 2024-03-23T07:57:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.271+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc151080 con 0x7f18dc159830 2024-03-23T07:57:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.271+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc151640 con 0x7f18dc159830 2024-03-23T07:57:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.272+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc151830 con 0x7f18dc159830 2024-03-23T07:57:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.273+0000 7f18d9600640 1 -- 172.21.15.105:0/1099746702 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f18d403d070 con 0x7f18dc159830 2024-03-23T07:57:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.274+0000 7f18d9600640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f18b806ac00 0x7f18b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.274+0000 7f18d9600640 1 -- 172.21.15.105:0/1099746702 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f18d40abfc0 con 0x7f18dc159830 2024-03-23T07:57:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.274+0000 7f18e0e00640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f18b806ac00 0x7f18b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.275+0000 7f18e0e00640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f18b806ac00 0x7f18b806d0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18cc001550 tx=0x7f18cc007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.279+0000 7f18d9600640 1 -- 172.21.15.105:0/1099746702 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f18d40df580 con 0x7f18dc159830 2024-03-23T07:57:21.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.471+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f18dc073390 con 0x7f18dc159830 2024-03-23T07:57:21.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.472+0000 7f18d9600640 1 -- 172.21.15.105:0/1099746702 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f18d4043020 con 0x7f18dc159830 2024-03-23T07:57:21.471 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f18b806ac00 msgr2=0x7f18b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f18b806ac00 0x7f18b806d0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18cc001550 tx=0x7f18cc007a40 comp rx=0 tx=0).stop 2024-03-23T07:57:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc159830 msgr2=0x7f18dc150440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc159830 0x7f18dc150440 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f18d4039040 tx=0x7f18d4004a40 comp rx=0 tx=0).stop 2024-03-23T07:57:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 shutdown_connections 2024-03-23T07:57:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18dc159830 0x7f18dc150440 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18dc0ce930 0x7f18dc14ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f18b806ac00 0x7f18b806d0c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 --2- 172.21.15.105:0/1099746702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18dc0ca7b0 0x7f18dc1490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.474+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 >> 172.21.15.105:0/1099746702 conn(0x7f18dc06a0b0 msgr2=0x7f18dc0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:21.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.475+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 shutdown_connections 2024-03-23T07:57:21.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.475+0000 7f18e3800640 1 -- 172.21.15.105:0/1099746702 wait complete. 2024-03-23T07:57:21.515 INFO:tasks.cephfs_test_runner:Starting test: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) MDS starts to handle client caps when it enters clientreplay stage. 2024-03-23T07:57:21.516 INFO:tasks.cephfs_test_runner:test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) 2024-03-23T07:57:21.516 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-03-23T07:57:21.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.731+0000 7f6117200640 1 Processor -- start 2024-03-23T07:57:21.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.731+0000 7f6117200640 1 -- start start 2024-03-23T07:57:21.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.733+0000 7f6117200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61180cb560 0x7f61180b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.733+0000 7f6117200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61181575b0 con 0x7f61181571d0 2024-03-23T07:57:21.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.733+0000 7f6117200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6118159f20 con 0x7f61180cdc00 2024-03-23T07:57:21.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.733+0000 7f6117200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611815a200 con 0x7f61180cb560 2024-03-23T07:57:21.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.733+0000 7f6116800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61180cb560 0x7f61180b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.733+0000 7f6116800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61180cb560 0x7f61180b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36260/0 (socket says 172.21.15.105:36260) 2024-03-23T07:57:21.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.733+0000 7f6116800640 1 -- 172.21.15.105:0/1434445447 learned_addr learned my addr 172.21.15.105:0/1434445447 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:21.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.734+0000 7f6116800640 1 -- 172.21.15.105:0/1434445447 >> v1:172.21.15.132:6789/0 conn(0x7f61180cdc00 legacy=0x7f61180cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:21.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.734+0000 7f6116800640 1 -- 172.21.15.105:0/1434445447 >> v1:172.21.15.105:6789/0 conn(0x7f61181571d0 legacy=0x7f611815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.734+0000 7f6114a00640 1 -- 172.21.15.105:0/1434445447 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062579109 0 0) 0x7f61181575b0 con 0x7f61181571d0 2024-03-23T07:57:21.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.734+0000 7f6116800640 1 -- 172.21.15.105:0/1434445447 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611815a4e0 con 0x7f61180cb560 2024-03-23T07:57:21.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.735+0000 7f6116800640 1 --2- 172.21.15.105:0/1434445447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180cb560 0x7f61180b35d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e50 tx=0x7f611002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50aa93919ad375e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:21.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.735+0000 7f6114a00640 1 -- 172.21.15.105:0/1434445447 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611002cbb0 con 0x7f61180cb560 2024-03-23T07:57:21.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6114a00640 1 -- 172.21.15.105:0/1434445447 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611002cd10 con 0x7f61180cb560 2024-03-23T07:57:21.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6114a00640 1 -- 172.21.15.105:0/1434445447 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110034610 con 0x7f61180cb560 2024-03-23T07:57:21.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6117200640 1 -- 172.21.15.105:0/1434445447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180cb560 msgr2=0x7f61180b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6117200640 1 --2- 172.21.15.105:0/1434445447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180cb560 0x7f61180b35d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e50 tx=0x7f611002c010 comp rx=0 tx=0).stop 2024-03-23T07:57:21.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6117200640 1 -- 172.21.15.105:0/1434445447 shutdown_connections 2024-03-23T07:57:21.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6117200640 1 --2- 172.21.15.105:0/1434445447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61180cb560 0x7f61180b35d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6117200640 1 -- 172.21.15.105:0/1434445447 >> 172.21.15.105:0/1434445447 conn(0x7f6118065df0 msgr2=0x7f6118069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:21.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6117200640 1 -- 172.21.15.105:0/1434445447 shutdown_connections 2024-03-23T07:57:21.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.736+0000 7f6117200640 1 -- 172.21.15.105:0/1434445447 wait complete. 2024-03-23T07:57:21.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.738+0000 7f6117200640 1 Processor -- start 2024-03-23T07:57:21.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.738+0000 7f6117200640 1 -- start start 2024-03-23T07:57:21.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.739+0000 7f6117200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180cb560 0x7f61180ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.739+0000 7f6117200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61180cdc00 0x7f61180af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.739+0000 7f6117200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61181571d0 0x7f61180b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.739+0000 7f6117200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180ae6a0 con 0x7f61181571d0 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.739+0000 7f6117200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180ae980 con 0x7f61180cdc00 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.739+0000 7f6117200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61180aec60 con 0x7f61180cb560 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.739+0000 7f6116800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61181571d0 0x7f61180b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61181571d0 0x7f61180b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36274/0 (socket says 172.21.15.105:36274) 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 -- 172.21.15.105:0/1327148180 learned_addr learned my addr 172.21.15.105:0/1327148180 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6115400640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61180cdc00 0x7f61180af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6115e00640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180cb560 0x7f61180ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 -- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180cb560 msgr2=0x7f61180ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180cb560 0x7f61180ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 -- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61180cdc00 msgr2=0x7f61180af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61180cdc00 0x7f61180af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 -- 172.21.15.105:0/1327148180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6110009b30 con 0x7f61181571d0 2024-03-23T07:57:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.740+0000 7f6116800640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61181571d0 0x7f61180b0060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6110036040 tx=0x7f6110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.741+0000 7f610e000640 1 -- 172.21.15.105:0/1327148180 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611003e450 con 0x7f61181571d0 2024-03-23T07:57:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.741+0000 7f610e000640 1 -- 172.21.15.105:0/1327148180 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611003e5d0 con 0x7f61181571d0 2024-03-23T07:57:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.741+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61180aef40 con 0x7f61181571d0 2024-03-23T07:57:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.741+0000 7f610e000640 1 -- 172.21.15.105:0/1327148180 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611003e980 con 0x7f61181571d0 2024-03-23T07:57:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.741+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6118141f10 con 0x7f61181571d0 2024-03-23T07:57:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.742+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61180af460 con 0x7f61181571d0 2024-03-23T07:57:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.743+0000 7f610e000640 1 -- 172.21.15.105:0/1327148180 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f611003eae0 con 0x7f61181571d0 2024-03-23T07:57:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.744+0000 7f610e000640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f60f806ac00 0x7f60f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.744+0000 7f610e000640 1 -- 172.21.15.105:0/1327148180 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f61100ac6b0 con 0x7f61181571d0 2024-03-23T07:57:21.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.744+0000 7f6115e00640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f60f806ac00 0x7f60f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:21.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.745+0000 7f6115e00640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f60f806ac00 0x7f60f806d0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6108000f60 tx=0x7f6108009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:21.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:21.749+0000 7f610e000640 1 -- 172.21.15.105:0/1327148180 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f61180af460 con 0x7f61181571d0 2024-03-23T07:57:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.018+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) v1 -- 0x7f61180af460 con 0x7f61181571d0 2024-03-23T07:57:22.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.187+0000 7f610e000640 1 -- 172.21.15.105:0/1327148180 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v77) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61180af460 con 0x7f61181571d0 2024-03-23T07:57:22.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f60f806ac00 msgr2=0x7f60f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f60f806ac00 0x7f60f806d0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6108000f60 tx=0x7f6108009040 comp rx=0 tx=0).stop 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61181571d0 msgr2=0x7f61180b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61181571d0 0x7f61180b0060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6110036040 tx=0x7f6110004a40 comp rx=0 tx=0).stop 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 shutdown_connections 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61181571d0 0x7f61180b0060 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61180cdc00 0x7f61180af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f60f806ac00 0x7f60f806d0c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 --2- 172.21.15.105:0/1327148180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61180cb560 0x7f61180ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 >> 172.21.15.105:0/1327148180 conn(0x7f6118065df0 msgr2=0x7f61180cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 shutdown_connections 2024-03-23T07:57:22.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.191+0000 7f6117200640 1 -- 172.21.15.105:0/1327148180 wait complete. 2024-03-23T07:57:22.211 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T07:57:22.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.415+0000 7f22d0200640 1 Processor -- start 2024-03-23T07:57:22.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.415+0000 7f22d0200640 1 -- start start 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.416+0000 7f22d0200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c81456a0 0x7f22c80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.416+0000 7f22d0200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f22c8070cd0 0x7f22c80cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.416+0000 7f22d0200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22c8145c20 0x7f22c80ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.416+0000 7f22d0200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c80ad890 con 0x7f22c8145c20 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.416+0000 7f22d0200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c8145a80 con 0x7f22c8070cd0 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.416+0000 7f22d0200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c806d390 con 0x7f22c81456a0 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22c8145c20 0x7f22c80ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22c8145c20 0x7f22c80ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36282/0 (socket says 172.21.15.105:36282) 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 -- 172.21.15.105:0/3476233579 learned_addr learned my addr 172.21.15.105:0/3476233579 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 -- 172.21.15.105:0/3476233579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c81456a0 msgr2=0x7f22c80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:22.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 --2- 172.21.15.105:0/3476233579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c81456a0 0x7f22c80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 -- 172.21.15.105:0/3476233579 >> v2:172.21.15.132:3300/0 conn(0x7f22c8070cd0 msgr2=0x7f22c80cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:22.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 --2- 172.21.15.105:0/3476233579 >> v2:172.21.15.132:3300/0 conn(0x7f22c8070cd0 0x7f22c80cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.417+0000 7f22ce200640 1 -- 172.21.15.105:0/3476233579 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c806d670 con 0x7f22c8145c20 2024-03-23T07:57:22.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.418+0000 7f22ce200640 1 --2- 172.21.15.105:0/3476233579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8145c20 0x7f22c80ab8b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cf570 tx=0x7f22c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29211b3ad4817be5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:22.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22c7e00640 1 -- 172.21.15.105:0/3476233579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0031640 con 0x7f22c8145c20 2024-03-23T07:57:22.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22c7e00640 1 -- 172.21.15.105:0/3476233579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c00317a0 con 0x7f22c8145c20 2024-03-23T07:57:22.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22c7e00640 1 -- 172.21.15.105:0/3476233579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0031b30 con 0x7f22c8145c20 2024-03-23T07:57:22.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 -- 172.21.15.105:0/3476233579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8145c20 msgr2=0x7f22c80ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:22.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 --2- 172.21.15.105:0/3476233579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8145c20 0x7f22c80ab8b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cf570 tx=0x7f22c002b8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 -- 172.21.15.105:0/3476233579 shutdown_connections 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 --2- 172.21.15.105:0/3476233579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8145c20 0x7f22c80ab8b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 --2- 172.21.15.105:0/3476233579 >> v2:172.21.15.132:3300/0 conn(0x7f22c8070cd0 0x7f22c80cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 --2- 172.21.15.105:0/3476233579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c81456a0 0x7f22c80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 -- 172.21.15.105:0/3476233579 >> 172.21.15.105:0/3476233579 conn(0x7f22c806a0b0 msgr2=0x7f22c806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 -- 172.21.15.105:0/3476233579 shutdown_connections 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.419+0000 7f22d0200640 1 -- 172.21.15.105:0/3476233579 wait complete. 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.421+0000 7f22d0200640 1 Processor -- start 2024-03-23T07:57:22.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.421+0000 7f22d0200640 1 -- start start 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22d0200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8070cd0 0x7f22c80b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22d0200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c81456a0 0x7f22c80b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22d0200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c8145c20 0x7f22c813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22d0200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80ae2c0 con 0x7f22c8070cd0 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22d0200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80af0e0 con 0x7f22c81456a0 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22d0200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80afd70 con 0x7f22c8145c20 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22cd800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8070cd0 0x7f22c80b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:22.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22cd800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8070cd0 0x7f22c80b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36298/0 (socket says 172.21.15.105:36298) 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.422+0000 7f22cd800640 1 -- 172.21.15.105:0/3093763556 learned_addr learned my addr 172.21.15.105:0/3093763556 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22cce00640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c81456a0 0x7f22c80b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22ce200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c8145c20 0x7f22c813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22cd800640 1 -- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c8145c20 msgr2=0x7f22c813c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22cd800640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c8145c20 0x7f22c813c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22cd800640 1 -- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c81456a0 msgr2=0x7f22c80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22cd800640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c81456a0 0x7f22c80b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22cd800640 1 -- 172.21.15.105:0/3093763556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00094d0 con 0x7f22c8070cd0 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22ce200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c8145c20 0x7f22c813c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22cd800640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8070cd0 0x7f22c80b42e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00ba50 tx=0x7f22bc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22c5600640 1 -- 172.21.15.105:0/3093763556 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00cbe0 con 0x7f22c8070cd0 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22c5600640 1 -- 172.21.15.105:0/3093763556 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc00cd60 con 0x7f22c8070cd0 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c80ae4d0 con 0x7f22c8070cd0 2024-03-23T07:57:22.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.423+0000 7f22c5600640 1 -- 172.21.15.105:0/3093763556 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc014680 con 0x7f22c8070cd0 2024-03-23T07:57:22.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.424+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c80aea90 con 0x7f22c8070cd0 2024-03-23T07:57:22.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.425+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c8073390 con 0x7f22c8070cd0 2024-03-23T07:57:22.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.426+0000 7f22c5600640 1 -- 172.21.15.105:0/3093763556 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f22bc01d3b0 con 0x7f22c8070cd0 2024-03-23T07:57:22.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.426+0000 7f22c5600640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22a806ac30 0x7f22a806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:22.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.427+0000 7f22c5600640 1 -- 172.21.15.105:0/3093763556 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f22bc089660 con 0x7f22c8070cd0 2024-03-23T07:57:22.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.427+0000 7f22cce00640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22a806ac30 0x7f22a806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:22.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.427+0000 7f22cce00640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22a806ac30 0x7f22a806d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22c815c7f0 tx=0x7f22b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:22.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.431+0000 7f22c5600640 1 -- 172.21.15.105:0/3093763556 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f22bc0544d0 con 0x7f22c8070cd0 2024-03-23T07:57:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.620+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f22c80ade90 con 0x7f22c8070cd0 2024-03-23T07:57:22.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.629+0000 7f22c5600640 1 -- 172.21.15.105:0/3093763556 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f22bc057af0 con 0x7f22c8070cd0 2024-03-23T07:57:22.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.631+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22a806ac30 msgr2=0x7f22a806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:22.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.631+0000 7f22d0200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22a806ac30 0x7f22a806d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22c815c7f0 tx=0x7f22b4008040 comp rx=0 tx=0).stop 2024-03-23T07:57:22.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.631+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8070cd0 msgr2=0x7f22c80b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.631+0000 7f22d0200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8070cd0 0x7f22c80b42e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00ba50 tx=0x7f22bc00bf10 comp rx=0 tx=0).stop 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 shutdown_connections 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22a806ac30 0x7f22a806d0f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22c8145c20 0x7f22c813c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c81456a0 0x7f22c80b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 --2- 172.21.15.105:0/3093763556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c8070cd0 0x7f22c80b42e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 >> 172.21.15.105:0/3093763556 conn(0x7f22c806a0b0 msgr2=0x7f22c806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 shutdown_connections 2024-03-23T07:57:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:22.632+0000 7f22d0200640 1 -- 172.21.15.105:0/3093763556 wait complete. 2024-03-23T07:57:22.648 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T07:57:22.648 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T07:57:22.754 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T07:57:22.876 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T07:57:22.876 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:22.876 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:22.933 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:22.934 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:22.985 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:22.985 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:23.036 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.248+0000 7f11fd200640 1 Processor -- start 2024-03-23T07:57:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.248+0000 7f11fd200640 1 -- start start 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.250+0000 7f11fd200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 0x7f11f80b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.250+0000 7f11fd200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80b16a0 con 0x7f11f806d390 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.250+0000 7f11fd200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80cbfb0 con 0x7f11f80adbc0 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.250+0000 7f11fd200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80cc290 con 0x7f11f80cbab0 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.250+0000 7f11f6800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f11f806d390 0x7f11f80b2aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47794/0 (socket says 172.21.15.105:47794) 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.250+0000 7f11f6800640 1 -- 172.21.15.105:0/40664043 learned_addr learned my addr 172.21.15.105:0/40664043 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.250+0000 7f11f5400640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 0x7f11f80b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117033503 0 0) 0x7f11f80b16a0 con 0x7f11f806d390 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11dc003520 con 0x7f11f806d390 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2547486002 0 0) 0x7f11dc003520 con 0x7f11f806d390 2024-03-23T07:57:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 msgr2=0x7f11f80b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 0x7f11f80b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 >> v1:172.21.15.132:6789/0 conn(0x7f11f80adbc0 legacy=0x7f11f80b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80b18f0 con 0x7f11f806d390 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f11f00019f0 con 0x7f11f806d390 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 >> v1:172.21.15.105:6789/0 conn(0x7f11f806d390 legacy=0x7f11f80b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11dc0039a0 0x7f11dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11dc0047c0 0x7f11dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 0x7f11dc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80cc290 con 0x7f11dc0039a0 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc00dd80 con 0x7f11dc0047c0 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc00df40 con 0x7f11f80cbab0 2024-03-23T07:57:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.251+0000 7f11f5e00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11dc0039a0 0x7f11dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f5400640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11dc0047c0 0x7f11dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f6800640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 0x7f11dc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f5e00640 1 -- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 msgr2=0x7f11dc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f5e00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 0x7f11dc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f5e00640 1 -- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11dc0047c0 msgr2=0x7f11dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f5e00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11dc0047c0 0x7f11dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f5e00640 1 -- 172.21.15.105:0/40664043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc003520 con 0x7f11dc0039a0 2024-03-23T07:57:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.252+0000 7f11f6800640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11f80cbab0 0x7f11dc005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.253+0000 7f11f5e00640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11dc0039a0 0x7f11dc003dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11e8001750 tx=0x7f11dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5c1b0d1e7a275c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.253+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80052d0 con 0x7f11dc0039a0 2024-03-23T07:57:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.253+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e802e050 con 0x7f11dc0039a0 2024-03-23T07:57:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.253+0000 7f11f4a00640 1 -- 172.21.15.105:0/40664043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80035f0 con 0x7f11dc0039a0 2024-03-23T07:57:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.254+0000 7f11fd200640 1 -- 172.21.15.105:0/40664043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11dc0039a0 msgr2=0x7f11dc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.254+0000 7f11fd200640 1 --2- 172.21.15.105:0/40664043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11dc0039a0 0x7f11dc003dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11e8001750 tx=0x7f11dc0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.254+0000 7f11f6800640 1 -- 172.21.15.105:0/40664043 reap_dead start 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.254+0000 7f11fd200640 1 -- 172.21.15.105:0/40664043 shutdown_connections 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.254+0000 7f11fd200640 1 -- 172.21.15.105:0/40664043 >> 172.21.15.105:0/40664043 conn(0x7f11f806a3d0 msgr2=0x7f11f80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.254+0000 7f11fd200640 1 -- 172.21.15.105:0/40664043 shutdown_connections 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.254+0000 7f11fd200640 1 -- 172.21.15.105:0/40664043 wait complete. 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.256+0000 7f11fd200640 1 Processor -- start 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.256+0000 7f11fd200640 1 -- start start 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11fd200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11dc0039a0 0x7f11f8140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11fd200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11f815c3b0 0x7f11f813ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11fd200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f815c830 0x7f11f80cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11fd200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81f67e0 con 0x7f11f815c3b0 2024-03-23T07:57:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11fd200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81f5e40 con 0x7f11f815c830 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11fd200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81f6120 con 0x7f11dc0039a0 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11f815c3b0 0x7f11f813ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11f815c3b0 0x7f11f813ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36316/0 (socket says 172.21.15.105:36316) 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 -- 172.21.15.105:0/1100198376 learned_addr learned my addr 172.21.15.105:0/1100198376 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 -- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11dc0039a0 msgr2=0x7f11f8140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5e00640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11dc0039a0 0x7f11f8140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f6800640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f815c830 0x7f11f80cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11dc0039a0 0x7f11f8140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 -- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f815c830 msgr2=0x7f11f80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f815c830 0x7f11f80cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f5400640 1 -- 172.21.15.105:0/1100198376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f81f6400 con 0x7f11f815c3b0 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.257+0000 7f11f6800640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f815c830 0x7f11f80cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.258+0000 7f11f5400640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11f815c3b0 0x7f11f813ae10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f11e4011600 tx=0x7f11e4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:23.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.258+0000 7f11ee000640 1 -- 172.21.15.105:0/1100198376 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400f920 con 0x7f11f815c3b0 2024-03-23T07:57:23.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.258+0000 7f11ee000640 1 -- 172.21.15.105:0/1100198376 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e400fa80 con 0x7f11f815c3b0 2024-03-23T07:57:23.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.258+0000 7f11ee000640 1 -- 172.21.15.105:0/1100198376 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400fdf0 con 0x7f11f815c3b0 2024-03-23T07:57:23.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.258+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f81f6c00 con 0x7f11f815c3b0 2024-03-23T07:57:23.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.258+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f813b350 con 0x7f11f815c3b0 2024-03-23T07:57:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.259+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f81f6a50 con 0x7f11f815c3b0 2024-03-23T07:57:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.260+0000 7f11ee000640 1 -- 172.21.15.105:0/1100198376 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11e4015460 con 0x7f11f815c3b0 2024-03-23T07:57:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.261+0000 7f11ee000640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f11d806ac00 0x7f11d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.261+0000 7f11ee000640 1 -- 172.21.15.105:0/1100198376 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f11e4089a50 con 0x7f11f815c3b0 2024-03-23T07:57:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.261+0000 7f11f5e00640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f11d806ac00 0x7f11d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.262+0000 7f11f5e00640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f11d806ac00 0x7f11d806d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f11e8033000 tx=0x7f11e8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.266+0000 7f11ee000640 1 -- 172.21.15.105:0/1100198376 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11e4054970 con 0x7f11f815c3b0 2024-03-23T07:57:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.461+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11f80b3160 con 0x7f11f815c3b0 2024-03-23T07:57:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.463+0000 7f11ee000640 1 -- 172.21.15.105:0/1100198376 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f11e4022090 con 0x7f11f815c3b0 2024-03-23T07:57:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:23.463 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f11d806ac00 msgr2=0x7f11d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f11d806ac00 0x7f11d806d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f11e8033000 tx=0x7f11e8009030 comp rx=0 tx=0).stop 2024-03-23T07:57:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11f815c3b0 msgr2=0x7f11f813ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11f815c3b0 0x7f11f813ae10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f11e4011600 tx=0x7f11e4011ac0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 shutdown_connections 2024-03-23T07:57:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11f815c830 0x7f11f80cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11f815c3b0 0x7f11f813ae10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f11d806ac00 0x7f11d806d0c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 --2- 172.21.15.105:0/1100198376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11dc0039a0 0x7f11f8140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 >> 172.21.15.105:0/1100198376 conn(0x7f11f806a3d0 msgr2=0x7f11f8141f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:23.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.466+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 shutdown_connections 2024-03-23T07:57:23.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.467+0000 7f11fd200640 1 -- 172.21.15.105:0/1100198376 wait complete. 2024-03-23T07:57:23.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:23.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:23.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.700+0000 7f5e03a00640 1 Processor -- start 2024-03-23T07:57:23.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.700+0000 7f5e03a00640 1 -- start start 2024-03-23T07:57:23.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.702+0000 7f5e03a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dfc06ce90 0x7f5dfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.702+0000 7f5e03a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc0cdba0 con 0x7f5dfc0add30 2024-03-23T07:57:23.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.702+0000 7f5e03a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc0cde80 con 0x7f5dfc0ce930 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.702+0000 7f5e03a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc0ae1c0 con 0x7f5dfc06ce90 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.702+0000 7f5e01a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dfc06ce90 0x7f5dfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.702+0000 7f5dfbe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5dfc0add30 0x7f5dfc0ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47808/0 (socket says 172.21.15.105:47808) 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.702+0000 7f5dfbe00640 1 -- 172.21.15.105:0/4289637005 learned_addr learned my addr 172.21.15.105:0/4289637005 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.703+0000 7f5dfb400640 1 -- 172.21.15.105:0/4289637005 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192471697 0 0) 0x7f5dfc0cdba0 con 0x7f5dfc0add30 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.703+0000 7f5dfb400640 1 -- 172.21.15.105:0/4289637005 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de4003520 con 0x7f5dfc0add30 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.703+0000 7f5e01a00640 1 -- 172.21.15.105:0/4289637005 >> v1:172.21.15.132:6789/0 conn(0x7f5dfc0ce930 legacy=0x7f5dfc0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.703+0000 7f5e01a00640 1 -- 172.21.15.105:0/4289637005 >> v1:172.21.15.105:6789/0 conn(0x7f5dfc0add30 legacy=0x7f5dfc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.703+0000 7f5e01a00640 1 -- 172.21.15.105:0/4289637005 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc0ce160 con 0x7f5dfc06ce90 2024-03-23T07:57:23.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.704+0000 7f5e01a00640 1 --2- 172.21.15.105:0/4289637005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc06ce90 0x7f5dfc06d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009d90 tx=0x7f5df402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f7b085ccee0362 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:23.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.704+0000 7f5dfb400640 1 -- 172.21.15.105:0/4289637005 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df402cd00 con 0x7f5dfc06ce90 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.704+0000 7f5dfb400640 1 -- 172.21.15.105:0/4289637005 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df402ce60 con 0x7f5dfc06ce90 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.704+0000 7f5dfb400640 1 -- 172.21.15.105:0/4289637005 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4034640 con 0x7f5dfc06ce90 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.705+0000 7f5e03a00640 1 -- 172.21.15.105:0/4289637005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc06ce90 msgr2=0x7f5dfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.705+0000 7f5e03a00640 1 --2- 172.21.15.105:0/4289637005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc06ce90 0x7f5dfc06d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009d90 tx=0x7f5df402bf50 comp rx=0 tx=0).stop 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.705+0000 7f5e03a00640 1 -- 172.21.15.105:0/4289637005 shutdown_connections 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.705+0000 7f5e03a00640 1 --2- 172.21.15.105:0/4289637005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc06ce90 0x7f5dfc06d300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.705+0000 7f5e03a00640 1 -- 172.21.15.105:0/4289637005 >> 172.21.15.105:0/4289637005 conn(0x7f5dfc06a0b0 msgr2=0x7f5dfc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.705+0000 7f5e03a00640 1 -- 172.21.15.105:0/4289637005 shutdown_connections 2024-03-23T07:57:23.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.705+0000 7f5e03a00640 1 -- 172.21.15.105:0/4289637005 wait complete. 2024-03-23T07:57:23.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.707+0000 7f5e03a00640 1 Processor -- start 2024-03-23T07:57:23.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.707+0000 7f5e03a00640 1 -- start start 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e03a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dfc06ce90 0x7f5dfc150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e03a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dfc0add30 0x7f5dfc1f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e03a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc0ce930 0x7f5dfc15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e03a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc1568d0 con 0x7f5dfc0ce930 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e03a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc156bb0 con 0x7f5dfc0add30 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e03a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc156e90 con 0x7f5dfc06ce90 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc0ce930 0x7f5dfc15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc0ce930 0x7f5dfc15cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36322/0 (socket says 172.21.15.105:36322) 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01a00640 1 -- 172.21.15.105:0/371680473 learned_addr learned my addr 172.21.15.105:0/371680473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01000640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dfc06ce90 0x7f5dfc150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5dfbe00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dfc0add30 0x7f5dfc1f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01a00640 1 -- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dfc06ce90 msgr2=0x7f5dfc150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dfc06ce90 0x7f5dfc150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01a00640 1 -- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dfc0add30 msgr2=0x7f5dfc1f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.708+0000 7f5e01a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dfc0add30 0x7f5dfc1f04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5e01a00640 1 -- 172.21.15.105:0/371680473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df4009a70 con 0x7f5dfc0ce930 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5dfbe00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dfc0add30 0x7f5dfc1f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5e01a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc0ce930 0x7f5dfc15cc60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009d90 tx=0x7f5df4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5df8c00640 1 -- 172.21.15.105:0/371680473 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4003b90 con 0x7f5dfc0ce930 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5df8c00640 1 -- 172.21.15.105:0/371680473 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df4003cf0 con 0x7f5dfc0ce930 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc1effb0 con 0x7f5dfc0ce930 2024-03-23T07:57:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5df8c00640 1 -- 172.21.15.105:0/371680473 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4004250 con 0x7f5dfc0ce930 2024-03-23T07:57:23.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.709+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc1efc10 con 0x7f5dfc0ce930 2024-03-23T07:57:23.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.710+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc073390 con 0x7f5dfc0ce930 2024-03-23T07:57:23.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.711+0000 7f5df8c00640 1 -- 172.21.15.105:0/371680473 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5df403b050 con 0x7f5dfc0ce930 2024-03-23T07:57:23.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.712+0000 7f5df8c00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5ddc06ac00 0x7f5ddc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:23.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.712+0000 7f5df8c00640 1 -- 172.21.15.105:0/371680473 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5df40abbb0 con 0x7f5dfc0ce930 2024-03-23T07:57:23.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.712+0000 7f5e01000640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5ddc06ac00 0x7f5ddc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:23.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.713+0000 7f5e01000640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5ddc06ac00 0x7f5ddc06d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007fd0 tx=0x7f5df0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:23.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.716+0000 7f5df8c00640 1 -- 172.21.15.105:0/371680473 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5df4076ab0 con 0x7f5dfc0ce930 2024-03-23T07:57:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.914+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5dfc06c540 con 0x7f5dfc0ce930 2024-03-23T07:57:23.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.914+0000 7f5df8c00640 1 -- 172.21.15.105:0/371680473 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5df40390a0 con 0x7f5dfc0ce930 2024-03-23T07:57:23.914 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5ddc06ac00 msgr2=0x7f5ddc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5ddc06ac00 0x7f5ddc06d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007fd0 tx=0x7f5df0009040 comp rx=0 tx=0).stop 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc0ce930 msgr2=0x7f5dfc15cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc0ce930 0x7f5dfc15cc60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009d90 tx=0x7f5df4004a40 comp rx=0 tx=0).stop 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 shutdown_connections 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dfc0ce930 0x7f5dfc15cc60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dfc0add30 0x7f5dfc1f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5ddc06ac00 0x7f5ddc06d0c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 --2- 172.21.15.105:0/371680473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dfc06ce90 0x7f5dfc150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 >> 172.21.15.105:0/371680473 conn(0x7f5dfc06a0b0 msgr2=0x7f5dfc15e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 shutdown_connections 2024-03-23T07:57:23.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:23.917+0000 7f5e03a00640 1 -- 172.21.15.105:0/371680473 wait complete. 2024-03-23T07:57:23.934 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:24.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.145+0000 7f2785000640 1 Processor -- start 2024-03-23T07:57:24.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.145+0000 7f2785000640 1 -- start start 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f2785000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f27800ce930 0x7f27800add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f2785000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800cd3e0 0x7f27800ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f2785000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f278006ce90 0x7f278006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f2785000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278006c760 con 0x7f278006ce90 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f2785000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27800b42d0 con 0x7f27800ce930 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f2785000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27800cf250 con 0x7f27800cd3e0 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f277e800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f278006ce90 0x7f278006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f277e800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f278006ce90 0x7f278006d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36326/0 (socket says 172.21.15.105:36326) 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f277e800640 1 -- 172.21.15.105:0/227080333 learned_addr learned my addr 172.21.15.105:0/227080333 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f277d400640 1 --2- 172.21.15.105:0/227080333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800cd3e0 0x7f27800ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.147+0000 7f277de00640 1 --2- 172.21.15.105:0/227080333 >> v2:172.21.15.132:3300/0 conn(0x7f27800ce930 0x7f27800add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.148+0000 7f277e800640 1 -- 172.21.15.105:0/227080333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800cd3e0 msgr2=0x7f27800ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.148+0000 7f277e800640 1 --2- 172.21.15.105:0/227080333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800cd3e0 0x7f27800ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.148+0000 7f277e800640 1 -- 172.21.15.105:0/227080333 >> v2:172.21.15.132:3300/0 conn(0x7f27800ce930 msgr2=0x7f27800add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.148+0000 7f277e800640 1 --2- 172.21.15.105:0/227080333 >> v2:172.21.15.132:3300/0 conn(0x7f27800ce930 0x7f27800add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.148+0000 7f277e800640 1 -- 172.21.15.105:0/227080333 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27800cdba0 con 0x7f278006ce90 2024-03-23T07:57:24.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.148+0000 7f277de00640 1 --2- 172.21.15.105:0/227080333 >> v2:172.21.15.132:3300/0 conn(0x7f27800ce930 0x7f27800add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:24.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.149+0000 7f277e800640 1 --2- 172.21.15.105:0/227080333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278006ce90 0x7f278006d320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f27800cf540 tx=0x7f277802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbcbba7029771a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:24.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.149+0000 7f277ca00640 1 -- 172.21.15.105:0/227080333 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277802ca70 con 0x7f278006ce90 2024-03-23T07:57:24.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.149+0000 7f277ca00640 1 -- 172.21.15.105:0/227080333 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277802cbd0 con 0x7f278006ce90 2024-03-23T07:57:24.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.149+0000 7f277ca00640 1 -- 172.21.15.105:0/227080333 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780343f0 con 0x7f278006ce90 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 -- 172.21.15.105:0/227080333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278006ce90 msgr2=0x7f278006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 --2- 172.21.15.105:0/227080333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278006ce90 0x7f278006d320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f27800cf540 tx=0x7f277802bed0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 -- 172.21.15.105:0/227080333 shutdown_connections 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 --2- 172.21.15.105:0/227080333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278006ce90 0x7f278006d320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 --2- 172.21.15.105:0/227080333 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800cd3e0 0x7f27800ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 --2- 172.21.15.105:0/227080333 >> v2:172.21.15.132:3300/0 conn(0x7f27800ce930 0x7f27800add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 -- 172.21.15.105:0/227080333 >> 172.21.15.105:0/227080333 conn(0x7f278006a0b0 msgr2=0x7f278006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:24.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 -- 172.21.15.105:0/227080333 shutdown_connections 2024-03-23T07:57:24.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.150+0000 7f2785000640 1 -- 172.21.15.105:0/227080333 wait complete. 2024-03-23T07:57:24.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.152+0000 7f2785000640 1 Processor -- start 2024-03-23T07:57:24.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.152+0000 7f2785000640 1 -- start start 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.153+0000 7f2785000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278006ce90 0x7f27800ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.153+0000 7f2785000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800cd3e0 0x7f278015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.153+0000 7f2785000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27800ce930 0x7f278014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.153+0000 7f2785000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278015d890 con 0x7f27800ce930 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.153+0000 7f2785000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27800ae950 con 0x7f27800cd3e0 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.153+0000 7f2785000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27800aec30 con 0x7f278006ce90 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27800ce930 0x7f278014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27800ce930 0x7f278014a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36328/0 (socket says 172.21.15.105:36328) 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 -- 172.21.15.105:0/4109863810 learned_addr learned my addr 172.21.15.105:0/4109863810 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277de00640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278006ce90 0x7f27800ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277d400640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800cd3e0 0x7f278015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 -- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278006ce90 msgr2=0x7f27800ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278006ce90 0x7f27800ae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 -- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800cd3e0 msgr2=0x7f278015d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800cd3e0 0x7f278015d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 -- 172.21.15.105:0/4109863810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778009b30 con 0x7f27800ce930 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f277e800640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27800ce930 0x7f278014a070 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2778036040 tx=0x7f2778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.154+0000 7f2776000640 1 -- 172.21.15.105:0/4109863810 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778009510 con 0x7f27800ce930 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.155+0000 7f2776000640 1 -- 172.21.15.105:0/4109863810 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277803e450 con 0x7f27800ce930 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.155+0000 7f2776000640 1 -- 172.21.15.105:0/4109863810 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277803e800 con 0x7f27800ce930 2024-03-23T07:57:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.155+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27800aef10 con 0x7f27800ce930 2024-03-23T07:57:24.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.155+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27800af4d0 con 0x7f27800ce930 2024-03-23T07:57:24.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.156+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27800af6c0 con 0x7f27800ce930 2024-03-23T07:57:24.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.157+0000 7f2776000640 1 -- 172.21.15.105:0/4109863810 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f277803e960 con 0x7f27800ce930 2024-03-23T07:57:24.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.158+0000 7f2776000640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f276006ac00 0x7f276006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.158+0000 7f2776000640 1 -- 172.21.15.105:0/4109863810 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f27780ac550 con 0x7f27800ce930 2024-03-23T07:57:24.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.158+0000 7f277de00640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f276006ac00 0x7f276006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.159+0000 7f277de00640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f276006ac00 0x7f276006d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2770006fd0 tx=0x7f2770008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:24.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.162+0000 7f2776000640 1 -- 172.21.15.105:0/4109863810 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2778077340 con 0x7f27800ce930 2024-03-23T07:57:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.358+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2780073390 con 0x7f27800ce930 2024-03-23T07:57:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.359+0000 7f2776000640 1 -- 172.21.15.105:0/4109863810 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f277807a960 con 0x7f27800ce930 2024-03-23T07:57:24.359 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f276006ac00 msgr2=0x7f276006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f276006ac00 0x7f276006d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2770006fd0 tx=0x7f2770008040 comp rx=0 tx=0).stop 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27800ce930 msgr2=0x7f278014a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27800ce930 0x7f278014a070 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2778036040 tx=0x7f2778004a40 comp rx=0 tx=0).stop 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 shutdown_connections 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27800ce930 0x7f278014a070 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800cd3e0 0x7f278015d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f276006ac00 0x7f276006d0c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 --2- 172.21.15.105:0/4109863810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278006ce90 0x7f27800ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 >> 172.21.15.105:0/4109863810 conn(0x7f278006a0b0 msgr2=0x7f2780152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 shutdown_connections 2024-03-23T07:57:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.362+0000 7f2785000640 1 -- 172.21.15.105:0/4109863810 wait complete. 2024-03-23T07:57:24.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:24.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.589+0000 7f47ffc00640 1 Processor -- start 2024-03-23T07:57:24.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.589+0000 7f47ffc00640 1 -- start start 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.590+0000 7f47ffc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 0x7f47f80add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.590+0000 7f47ffc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47f80cd3e0 0x7f47f80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.590+0000 7f47ffc00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f80cf250 con 0x7f47f806ce90 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.590+0000 7f47ffc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f806c760 con 0x7f47f80cd3e0 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.590+0000 7f47ffc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80b42d0 con 0x7f47f80ce930 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47fdc00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f47f806ce90 0x7f47f815e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47818/0 (socket says 172.21.15.105:47818) 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47fdc00640 1 -- 172.21.15.105:0/1931980363 learned_addr learned my addr 172.21.15.105:0/1931980363 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7e00640 1 --2- 172.21.15.105:0/1931980363 >> v2:172.21.15.132:3300/0 conn(0x7f47f80cd3e0 0x7f47f80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47fd200640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 0x7f47f80add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559938808 0 0) 0x7f47f80cf250 con 0x7f47f806ce90 2024-03-23T07:57:24.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e0003520 con 0x7f47f806ce90 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 206254165 0 0) 0x7f47e0003520 con 0x7f47f806ce90 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 msgr2=0x7f47f80add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 0x7f47f80add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 >> v2:172.21.15.132:3300/0 conn(0x7f47f80cd3e0 msgr2=0x7f47f80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 --2- 172.21.15.105:0/1931980363 >> v2:172.21.15.132:3300/0 conn(0x7f47f80cd3e0 0x7f47f80ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80cdba0 con 0x7f47f806ce90 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.591+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f47f00019f0 con 0x7f47f806ce90 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 >> v1:172.21.15.105:6789/0 conn(0x7f47f806ce90 legacy=0x7f47f815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7400640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47e0003970 0x7f47e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7400640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 0x7f47e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7400640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f806c760 con 0x7f47e0004d00 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80b42d0 con 0x7f47e0003970 2024-03-23T07:57:24.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000ba30 con 0x7f47f80ce930 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fdc00640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fd200640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47e0003970 0x7f47e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47f7e00640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 0x7f47e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fdc00640 1 -- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 msgr2=0x7f47e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fdc00640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f80ce930 0x7f47e00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fdc00640 1 -- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47e0003970 msgr2=0x7f47e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fdc00640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47e0003970 0x7f47e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fdc00640 1 -- 172.21.15.105:0/1931980363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0003520 con 0x7f47e0004d00 2024-03-23T07:57:24.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.592+0000 7f47fd200640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47e0003970 0x7f47e0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:24.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.593+0000 7f47fdc00640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47f000a890 tx=0x7f47e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1837423ac6ddb08c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0034630 con 0x7f47e0004d00 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f002f070 con 0x7f47e0004d00 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47f7400640 1 -- 172.21.15.105:0/1931980363 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00349f0 con 0x7f47e0004d00 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47ffc00640 1 -- 172.21.15.105:0/1931980363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 msgr2=0x7f47e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47ffc00640 1 --2- 172.21.15.105:0/1931980363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47f000a890 tx=0x7f47e00057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47fdc00640 1 -- 172.21.15.105:0/1931980363 reap_dead start 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47ffc00640 1 -- 172.21.15.105:0/1931980363 shutdown_connections 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47ffc00640 1 -- 172.21.15.105:0/1931980363 >> 172.21.15.105:0/1931980363 conn(0x7f47f806a0b0 msgr2=0x7f47f806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47ffc00640 1 -- 172.21.15.105:0/1931980363 shutdown_connections 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.594+0000 7f47ffc00640 1 -- 172.21.15.105:0/1931980363 wait complete. 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.596+0000 7f47ffc00640 1 Processor -- start 2024-03-23T07:57:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.596+0000 7f47ffc00640 1 -- start start 2024-03-23T07:57:24.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47ffc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47f80ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47ffc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f815d0e0 0x7f47f80ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47ffc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f8155660 0x7f47f8154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47ffc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f81f2160 con 0x7f47e0004d00 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47ffc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f81f2440 con 0x7f47f8155660 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47ffc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f81f2720 con 0x7f47f815d0e0 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47fd200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47f80ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47fd200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47f80ac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36356/0 (socket says 172.21.15.105:36356) 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.597+0000 7f47fd200640 1 -- 172.21.15.105:0/864142058 learned_addr learned my addr 172.21.15.105:0/864142058 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47fdc00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f8155660 0x7f47f8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47f7e00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f815d0e0 0x7f47f80ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47fd200640 1 -- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f815d0e0 msgr2=0x7f47f80ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47fd200640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f815d0e0 0x7f47f80ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47fd200640 1 -- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f8155660 msgr2=0x7f47f8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47fd200640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f8155660 0x7f47f8154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47fd200640 1 -- 172.21.15.105:0/864142058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f81f2a00 con 0x7f47e0004d00 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47fd200640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47f80ac670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00e7e0 tx=0x7f47ec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:24.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47f4c00640 1 -- 172.21.15.105:0/864142058 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00c6d0 con 0x7f47e0004d00 2024-03-23T07:57:24.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47f4c00640 1 -- 172.21.15.105:0/864142058 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec00c830 con 0x7f47e0004d00 2024-03-23T07:57:24.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.598+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f8155af0 con 0x7f47e0004d00 2024-03-23T07:57:24.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.599+0000 7f47f4c00640 1 -- 172.21.15.105:0/864142058 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00cbe0 con 0x7f47e0004d00 2024-03-23T07:57:24.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.599+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f81f2ce0 con 0x7f47e0004d00 2024-03-23T07:57:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.600+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f80acbb0 con 0x7f47e0004d00 2024-03-23T07:57:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.601+0000 7f47f4c00640 1 -- 172.21.15.105:0/864142058 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f47ec00cd40 con 0x7f47e0004d00 2024-03-23T07:57:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.601+0000 7f47f4c00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d806ac00 0x7f47d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.602+0000 7f47f4c00640 1 -- 172.21.15.105:0/864142058 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f47ec089490 con 0x7f47e0004d00 2024-03-23T07:57:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.602+0000 7f47f7e00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d806ac00 0x7f47d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.602+0000 7f47f7e00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d806ac00 0x7f47d806d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47f815d500 tx=0x7f47e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.606+0000 7f47f4c00640 1 -- 172.21.15.105:0/864142058 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f47ec054320 con 0x7f47e0004d00 2024-03-23T07:57:24.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.783+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47f8073390 con 0x7f47e0004d00 2024-03-23T07:57:24.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.784+0000 7f47f4c00640 1 -- 172.21.15.105:0/864142058 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47ec057940 con 0x7f47e0004d00 2024-03-23T07:57:24.784 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d806ac00 msgr2=0x7f47d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d806ac00 0x7f47d806d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47f815d500 tx=0x7f47e4008040 comp rx=0 tx=0).stop 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 msgr2=0x7f47f80ac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47f80ac670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00e7e0 tx=0x7f47ec00eca0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 shutdown_connections 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f8155660 0x7f47f8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f47d806ac00 0x7f47d806d0c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47f815d0e0 0x7f47f80ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 --2- 172.21.15.105:0/864142058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47e0004d00 0x7f47f80ac670 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 >> 172.21.15.105:0/864142058 conn(0x7f47f806a0b0 msgr2=0x7f47f81586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:24.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 shutdown_connections 2024-03-23T07:57:24.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:24.787+0000 7f47ffc00640 1 -- 172.21.15.105:0/864142058 wait complete. 2024-03-23T07:57:24.804 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.011+0000 7f2fb6800640 1 Processor -- start 2024-03-23T07:57:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.012+0000 7f2fb6800640 1 -- start start 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.013+0000 7f2fb6800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2fb0159830 0x7f2fb00cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2fb6800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb0159c10 con 0x7f2fb00ce930 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2fb6800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb00cd840 con 0x7f2fb00ca7b0 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2fb6800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb006de50 con 0x7f2fb0159830 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2faf400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2fb0159830 0x7f2fb00cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2faf400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2fb0159830 0x7f2fb00cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36366/0 (socket says 172.21.15.105:36366) 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2faf400640 1 -- 172.21.15.105:0/2175338314 learned_addr learned my addr 172.21.15.105:0/2175338314 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2faf400640 1 -- 172.21.15.105:0/2175338314 >> v1:172.21.15.132:6789/0 conn(0x7f2fb00ca7b0 legacy=0x7f2fb00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2faf400640 1 -- 172.21.15.105:0/2175338314 >> v1:172.21.15.105:6789/0 conn(0x7f2fb00ce930 legacy=0x7f2fb00b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.014+0000 7f2faf400640 1 -- 172.21.15.105:0/2175338314 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00ccc30 con 0x7f2fb0159830 2024-03-23T07:57:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.015+0000 7f2faf400640 1 --2- 172.21.15.105:0/2175338314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb0159830 0x7f2fb00cc3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000bb40 tx=0x7f2fa002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f8453ef336a80d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.016+0000 7f2faea00640 1 -- 172.21.15.105:0/2175338314 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0036440 con 0x7f2fb0159830 2024-03-23T07:57:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.016+0000 7f2faea00640 1 -- 172.21.15.105:0/2175338314 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa00365a0 con 0x7f2fb0159830 2024-03-23T07:57:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.016+0000 7f2faea00640 1 -- 172.21.15.105:0/2175338314 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0036930 con 0x7f2fb0159830 2024-03-23T07:57:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.016+0000 7f2fb6800640 1 -- 172.21.15.105:0/2175338314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb0159830 msgr2=0x7f2fb00cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.016+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2175338314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb0159830 0x7f2fb00cc3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000bb40 tx=0x7f2fa002e170 comp rx=0 tx=0).stop 2024-03-23T07:57:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.017+0000 7f2fb6800640 1 -- 172.21.15.105:0/2175338314 shutdown_connections 2024-03-23T07:57:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.017+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2175338314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb0159830 0x7f2fb00cc3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.017+0000 7f2fb6800640 1 -- 172.21.15.105:0/2175338314 >> 172.21.15.105:0/2175338314 conn(0x7f2fb006a0b0 msgr2=0x7f2fb006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.017+0000 7f2fb6800640 1 -- 172.21.15.105:0/2175338314 shutdown_connections 2024-03-23T07:57:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.017+0000 7f2fb6800640 1 -- 172.21.15.105:0/2175338314 wait complete. 2024-03-23T07:57:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.018+0000 7f2fb6800640 1 Processor -- start 2024-03-23T07:57:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.018+0000 7f2fb6800640 1 -- start start 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.019+0000 7f2fb6800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb00ca7b0 0x7f2fb015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.019+0000 7f2fb6800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb00ce930 0x7f2fb014ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.019+0000 7f2fb6800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb0159830 0x7f2fb015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.019+0000 7f2fb6800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00b0280 con 0x7f2fb00ce930 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.019+0000 7f2fb6800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb015a9b0 con 0x7f2fb0159830 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.019+0000 7f2fb6800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb015ac90 con 0x7f2fb00ca7b0 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb00ce930 0x7f2fb014ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb00ce930 0x7f2fb014ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36382/0 (socket says 172.21.15.105:36382) 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 -- 172.21.15.105:0/2361240793 learned_addr learned my addr 172.21.15.105:0/2361240793 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 -- 172.21.15.105:0/2361240793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb00ca7b0 msgr2=0x7f2fb015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb00ca7b0 0x7f2fb015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 -- 172.21.15.105:0/2361240793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb0159830 msgr2=0x7f2fb015a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb0159830 0x7f2fb015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 -- 172.21.15.105:0/2361240793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa000b770 con 0x7f2fb00ce930 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.020+0000 7f2fa6a00640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb00ce930 0x7f2fb014ac40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c430 tx=0x7f2f9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.021+0000 7f2fa7e00640 1 -- 172.21.15.105:0/2361240793 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98011790 con 0x7f2fb00ce930 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.021+0000 7f2fa7e00640 1 -- 172.21.15.105:0/2361240793 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f980118f0 con 0x7f2fb00ce930 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.021+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb015af70 con 0x7f2fb00ce930 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.021+0000 7f2fa7e00640 1 -- 172.21.15.105:0/2361240793 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98011ce0 con 0x7f2fb00ce930 2024-03-23T07:57:25.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.021+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb015b530 con 0x7f2fb00ce930 2024-03-23T07:57:25.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.022+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb015b750 con 0x7f2fb00ce930 2024-03-23T07:57:25.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.023+0000 7f2fa7e00640 1 -- 172.21.15.105:0/2361240793 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2f9801c460 con 0x7f2fb00ce930 2024-03-23T07:57:25.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.024+0000 7f2fa7e00640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f8c06ac00 0x7f2f8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.024+0000 7f2fa7e00640 1 -- 172.21.15.105:0/2361240793 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2f980895c0 con 0x7f2fb00ce930 2024-03-23T07:57:25.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.024+0000 7f2faf400640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f8c06ac00 0x7f2f8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:25.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.025+0000 7f2faf400640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f8c06ac00 0x7f2f8c06d0c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0038000 tx=0x7f2fa0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:25.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.029+0000 7f2fa7e00640 1 -- 172.21.15.105:0/2361240793 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2f98054450 con 0x7f2fb00ce930 2024-03-23T07:57:25.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.227+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2fb0073390 con 0x7f2fb00ce930 2024-03-23T07:57:25.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.228+0000 7f2fa7e00640 1 -- 172.21.15.105:0/2361240793 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2f9805ce60 con 0x7f2fb00ce930 2024-03-23T07:57:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:25.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f8c06ac00 msgr2=0x7f2f8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f8c06ac00 0x7f2f8c06d0c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0038000 tx=0x7f2fa0008040 comp rx=0 tx=0).stop 2024-03-23T07:57:25.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb00ce930 msgr2=0x7f2fb014ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb00ce930 0x7f2fb014ac40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c430 tx=0x7f2f9800c8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 shutdown_connections 2024-03-23T07:57:25.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb0159830 0x7f2fb015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb00ce930 0x7f2fb014ac40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2f8c06ac00 0x7f2f8c06d0c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 --2- 172.21.15.105:0/2361240793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb00ca7b0 0x7f2fb015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.230+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 >> 172.21.15.105:0/2361240793 conn(0x7f2fb006a0b0 msgr2=0x7f2fb0157940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:25.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.231+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 shutdown_connections 2024-03-23T07:57:25.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.231+0000 7f2fb6800640 1 -- 172.21.15.105:0/2361240793 wait complete. 2024-03-23T07:57:25.248 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T07:57:25.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.455+0000 7f1a70800640 1 Processor -- start 2024-03-23T07:57:25.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.455+0000 7f1a70800640 1 -- start start 2024-03-23T07:57:25.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a70800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a680cb660 0x7f1a680cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a70800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a6806c2b0 0x7f1a6806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a70800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6815ae60 con 0x7f1a680ce930 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a70800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6815bae0 con 0x7f1a680cb660 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a70800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6815b140 con 0x7f1a6806c2b0 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a6d400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a680cb660 0x7f1a680cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a6d400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a680cb660 0x7f1a680cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36396/0 (socket says 172.21.15.105:36396) 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.457+0000 7f1a6d400640 1 -- 172.21.15.105:0/2541997690 learned_addr learned my addr 172.21.15.105:0/2541997690 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:25.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.458+0000 7f1a6d400640 1 -- 172.21.15.105:0/2541997690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a6806c2b0 msgr2=0x7f1a6806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:25.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.458+0000 7f1a6d400640 1 --2- 172.21.15.105:0/2541997690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a6806c2b0 0x7f1a6806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.458+0000 7f1a6d400640 1 -- 172.21.15.105:0/2541997690 >> v1:172.21.15.132:6789/0 conn(0x7f1a680ce930 legacy=0x7f1a680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:25.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.458+0000 7f1a6d400640 1 -- 172.21.15.105:0/2541997690 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6815b420 con 0x7f1a680cb660 2024-03-23T07:57:25.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.459+0000 7f1a6d400640 1 --2- 172.21.15.105:0/2541997690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a680cb660 0x7f1a680cbae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009870 tx=0x7f1a5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58845c4951a6f49 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:25.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.459+0000 7f1a6ca00640 1 -- 172.21.15.105:0/2541997690 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58031640 con 0x7f1a680cb660 2024-03-23T07:57:25.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.459+0000 7f1a6ca00640 1 -- 172.21.15.105:0/2541997690 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a580317a0 con 0x7f1a680cb660 2024-03-23T07:57:25.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.459+0000 7f1a6ca00640 1 -- 172.21.15.105:0/2541997690 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58031b30 con 0x7f1a680cb660 2024-03-23T07:57:25.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 -- 172.21.15.105:0/2541997690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a680cb660 msgr2=0x7f1a680cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 --2- 172.21.15.105:0/2541997690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a680cb660 0x7f1a680cbae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009870 tx=0x7f1a5802b8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 -- 172.21.15.105:0/2541997690 shutdown_connections 2024-03-23T07:57:25.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 --2- 172.21.15.105:0/2541997690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a6806c2b0 0x7f1a6806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 --2- 172.21.15.105:0/2541997690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a680cb660 0x7f1a680cbae0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 -- 172.21.15.105:0/2541997690 >> 172.21.15.105:0/2541997690 conn(0x7f1a6806a0b0 msgr2=0x7f1a6806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:25.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 -- 172.21.15.105:0/2541997690 shutdown_connections 2024-03-23T07:57:25.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.460+0000 7f1a70800640 1 -- 172.21.15.105:0/2541997690 wait complete. 2024-03-23T07:57:25.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.462+0000 7f1a70800640 1 Processor -- start 2024-03-23T07:57:25.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.462+0000 7f1a70800640 1 -- start start 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a70800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6806c2b0 0x7f1a6814dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a70800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a680cb660 0x7f1a6806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a70800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a680ce930 0x7f1a680b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a70800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680ae390 con 0x7f1a6806c2b0 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a70800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68157300 con 0x7f1a680ce930 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a70800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680ae630 con 0x7f1a680cb660 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6806c2b0 0x7f1a6814dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6806c2b0 0x7f1a6814dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36404/0 (socket says 172.21.15.105:36404) 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.463+0000 7f1a6de00640 1 -- 172.21.15.105:0/2276911520 learned_addr learned my addr 172.21.15.105:0/2276911520 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:25.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6d400640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a680cb660 0x7f1a6806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6de00640 1 -- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a680cb660 msgr2=0x7f1a6806c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6de00640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a680cb660 0x7f1a6806c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6de00640 1 -- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a680ce930 msgr2=0x7f1a680b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6de00640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a680ce930 0x7f1a680b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6de00640 1 -- 172.21.15.105:0/2276911520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580094d0 con 0x7f1a6806c2b0 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6d400640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a680cb660 0x7f1a6806c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a6de00640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6806c2b0 0x7f1a6814dd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c410 tx=0x7f1a5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a62000640 1 -- 172.21.15.105:0/2276911520 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0035f0 con 0x7f1a6806c2b0 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a62000640 1 -- 172.21.15.105:0/2276911520 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c003750 con 0x7f1a6806c2b0 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a68155280 con 0x7f1a6806c2b0 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a62000640 1 -- 172.21.15.105:0/2276911520 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c003b40 con 0x7f1a6806c2b0 2024-03-23T07:57:25.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.464+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a681575e0 con 0x7f1a6806c2b0 2024-03-23T07:57:25.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.466+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a68073390 con 0x7f1a6806c2b0 2024-03-23T07:57:25.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.466+0000 7f1a62000640 1 -- 172.21.15.105:0/2276911520 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1a5c003ca0 con 0x7f1a6806c2b0 2024-03-23T07:57:25.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.467+0000 7f1a62000640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:25.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.467+0000 7f1a62000640 1 -- 172.21.15.105:0/2276911520 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1a5c088cf0 con 0x7f1a6806c2b0 2024-03-23T07:57:25.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.468+0000 7f1a6d400640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:25.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.468+0000 7f1a6d400640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a58036000 tx=0x7f1a5803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:25.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.472+0000 7f1a62000640 1 -- 172.21.15.105:0/2276911520 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1a5c08d070 con 0x7f1a6806c2b0 2024-03-23T07:57:25.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.679+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1a6806dbd0 con 0x7f1a6806c2b0 2024-03-23T07:57:25.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.680+0000 7f1a62000640 1 -- 172.21.15.105:0/2276911520 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1a5c017020 con 0x7f1a6806c2b0 2024-03-23T07:57:25.680 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1a4c06ac00 msgr2=0x7f1a4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a58036000 tx=0x7f1a5803c040 comp rx=0 tx=0).stop 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6806c2b0 msgr2=0x7f1a6814dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6806c2b0 0x7f1a6814dd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c410 tx=0x7f1a5c00c8d0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 shutdown_connections 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a680ce930 0x7f1a680b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1a4c06ac00 0x7f1a4c06d0c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a680cb660 0x7f1a6806c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 --2- 172.21.15.105:0/2276911520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6806c2b0 0x7f1a6814dd20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 >> 172.21.15.105:0/2276911520 conn(0x7f1a6806a0b0 msgr2=0x7f1a680707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:25.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 shutdown_connections 2024-03-23T07:57:25.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:25.683+0000 7f1a70800640 1 -- 172.21.15.105:0/2276911520 wait complete. 2024-03-23T07:57:25.700 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-23T07:57:25.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:25.700 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-23T07:57:25.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:57:25.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:25.792 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T07:57:25.842 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:25.842 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:25.899 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:25.899 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:25.950 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:25.950 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:26.000 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:26.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.220+0000 7fcb2de00640 1 Processor -- start 2024-03-23T07:57:26.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.220+0000 7fcb2de00640 1 -- start start 2024-03-23T07:57:26.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.222+0000 7fcb2de00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb2815cab0 0x7fcb280b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.222+0000 7fcb2de00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcb280ceeb0 0x7fcb280cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.222+0000 7fcb2de00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb280cf850 con 0x7fcb280b4120 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.222+0000 7fcb2de00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2815ce90 con 0x7fcb2815cab0 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.222+0000 7fcb2de00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb280cddf0 con 0x7fcb280ceeb0 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.222+0000 7fcb2ca00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb2815cab0 0x7fcb280b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.223+0000 7fcb2ca00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb2815cab0 0x7fcb280b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36418/0 (socket says 172.21.15.105:36418) 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.223+0000 7fcb2ca00640 1 -- 172.21.15.105:0/1026158380 learned_addr learned my addr 172.21.15.105:0/1026158380 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.223+0000 7fcb27e00640 1 --2- 172.21.15.105:0/1026158380 >> v2:172.21.15.132:3300/0 conn(0x7fcb280ceeb0 0x7fcb280cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.223+0000 7fcb2ca00640 1 -- 172.21.15.105:0/1026158380 >> v2:172.21.15.132:3300/0 conn(0x7fcb280ceeb0 msgr2=0x7fcb280cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.223+0000 7fcb2ca00640 1 --2- 172.21.15.105:0/1026158380 >> v2:172.21.15.132:3300/0 conn(0x7fcb280ceeb0 0x7fcb280cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.223+0000 7fcb2ca00640 1 -- 172.21.15.105:0/1026158380 >> v1:172.21.15.132:6789/0 conn(0x7fcb280b4120 legacy=0x7fcb280b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:26.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.223+0000 7fcb2ca00640 1 -- 172.21.15.105:0/1026158380 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280cd570 con 0x7fcb2815cab0 2024-03-23T07:57:26.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.224+0000 7fcb2ca00640 1 --2- 172.21.15.105:0/1026158380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b8050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb280d3750 tx=0x7fcb1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45a02ead18672e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.224+0000 7fcb27400640 1 -- 172.21.15.105:0/1026158380 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18031870 con 0x7fcb2815cab0 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.224+0000 7fcb27400640 1 -- 172.21.15.105:0/1026158380 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb180319d0 con 0x7fcb2815cab0 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.224+0000 7fcb27400640 1 -- 172.21.15.105:0/1026158380 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18031d60 con 0x7fcb2815cab0 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 -- 172.21.15.105:0/1026158380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 msgr2=0x7fcb280b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 --2- 172.21.15.105:0/1026158380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b8050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb280d3750 tx=0x7fcb1802b910 comp rx=0 tx=0).stop 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 -- 172.21.15.105:0/1026158380 shutdown_connections 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 --2- 172.21.15.105:0/1026158380 >> v2:172.21.15.132:3300/0 conn(0x7fcb280ceeb0 0x7fcb280cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 --2- 172.21.15.105:0/1026158380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b8050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 -- 172.21.15.105:0/1026158380 >> 172.21.15.105:0/1026158380 conn(0x7fcb2806ae90 msgr2=0x7fcb2806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 -- 172.21.15.105:0/1026158380 shutdown_connections 2024-03-23T07:57:26.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.225+0000 7fcb2de00640 1 -- 172.21.15.105:0/1026158380 wait complete. 2024-03-23T07:57:26.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.227+0000 7fcb2de00640 1 Processor -- start 2024-03-23T07:57:26.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.227+0000 7fcb2de00640 1 -- start start 2024-03-23T07:57:26.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb280b4120 0x7fcb2814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb280ceeb0 0x7fcb281f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2de00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2815e580 con 0x7fcb2815cab0 2024-03-23T07:57:26.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb281f2060 con 0x7fcb280ceeb0 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb281f6040 con 0x7fcb280b4120 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36434/0 (socket says 172.21.15.105:36434) 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2d400640 1 -- 172.21.15.105:0/19227520 learned_addr learned my addr 172.21.15.105:0/19227520 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.228+0000 7fcb2ca00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb280b4120 0x7fcb2814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb27e00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb280ceeb0 0x7fcb281f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2d400640 1 -- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb280b4120 msgr2=0x7fcb2814d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2d400640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb280b4120 0x7fcb2814d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2d400640 1 -- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb280ceeb0 msgr2=0x7fcb281f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2d400640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb280ceeb0 0x7fcb281f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2d400640 1 -- 172.21.15.105:0/19227520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180094d0 con 0x7fcb2815cab0 2024-03-23T07:57:26.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2d400640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b6920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000c780 tx=0x7fcb2000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb24c00640 1 -- 172.21.15.105:0/19227520 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20003d80 con 0x7fcb2815cab0 2024-03-23T07:57:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb24c00640 1 -- 172.21.15.105:0/19227520 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2000ce90 con 0x7fcb2815cab0 2024-03-23T07:57:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb281f6320 con 0x7fcb2815cab0 2024-03-23T07:57:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb24c00640 1 -- 172.21.15.105:0/19227520 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb200052b0 con 0x7fcb2815cab0 2024-03-23T07:57:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.229+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2815e210 con 0x7fcb2815cab0 2024-03-23T07:57:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.231+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2815e730 con 0x7fcb2815cab0 2024-03-23T07:57:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.231+0000 7fcb24c00640 1 -- 172.21.15.105:0/19227520 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcb20005410 con 0x7fcb2815cab0 2024-03-23T07:57:26.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.232+0000 7fcb24c00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcb0406ac00 0x7fcb0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.232+0000 7fcb24c00640 1 -- 172.21.15.105:0/19227520 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcb20088fd0 con 0x7fcb2815cab0 2024-03-23T07:57:26.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.233+0000 7fcb2ca00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcb0406ac00 0x7fcb0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.233+0000 7fcb2ca00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcb0406ac00 0x7fcb0406d0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcb18036000 tx=0x7fcb1803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.237+0000 7fcb24c00640 1 -- 172.21.15.105:0/19227520 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcb2815e730 con 0x7fcb2815cab0 2024-03-23T07:57:26.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.432+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb2804e030 con 0x7fcb2815cab0 2024-03-23T07:57:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.434+0000 7fcb24c00640 1 -- 172.21.15.105:0/19227520 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7fcb20017070 con 0x7fcb2815cab0 2024-03-23T07:57:26.434 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:26.434 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcb0406ac00 msgr2=0x7fcb0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcb0406ac00 0x7fcb0406d0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcb18036000 tx=0x7fcb1803c040 comp rx=0 tx=0).stop 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 msgr2=0x7fcb280b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b6920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000c780 tx=0x7fcb2000cc40 comp rx=0 tx=0).stop 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 shutdown_connections 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2815cab0 0x7fcb280b6920 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb280ceeb0 0x7fcb281f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcb0406ac00 0x7fcb0406d0c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 --2- 172.21.15.105:0/19227520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb280b4120 0x7fcb2814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 >> 172.21.15.105:0/19227520 conn(0x7fcb2806ae90 msgr2=0x7fcb281602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:26.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 shutdown_connections 2024-03-23T07:57:26.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.437+0000 7fcb2de00640 1 -- 172.21.15.105:0/19227520 wait complete. 2024-03-23T07:57:26.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:26.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:26.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.668+0000 7f90f3200640 1 Processor -- start 2024-03-23T07:57:26.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.668+0000 7f90f3200640 1 -- start start 2024-03-23T07:57:26.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.669+0000 7f90f3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f90ec0ce930 0x7f90ec0add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.669+0000 7f90f3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 0x7f90ec06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.669+0000 7f90f3200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ec0cdba0 con 0x7f90ec0cd3e0 2024-03-23T07:57:26.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.669+0000 7f90f3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ec06c760 con 0x7f90ec0ce930 2024-03-23T07:57:26.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.670+0000 7f90f3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0b42d0 con 0x7f90ec06ce90 2024-03-23T07:57:26.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.670+0000 7f90e3e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f90ec0cd3e0 0x7f90ec0ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47848/0 (socket says 172.21.15.105:47848) 2024-03-23T07:57:26.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.670+0000 7f90e3e00640 1 -- 172.21.15.105:0/3871660876 learned_addr learned my addr 172.21.15.105:0/3871660876 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:26.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.670+0000 7f90f1200640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 0x7f90ec06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.670+0000 7f90ebe00640 1 --2- 172.21.15.105:0/3871660876 >> v2:172.21.15.132:3300/0 conn(0x7f90ec0ce930 0x7f90ec0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.670+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708338518 0 0) 0x7f90ec0cdba0 con 0x7f90ec0cd3e0 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.670+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90cc003520 con 0x7f90ec0cd3e0 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729432905 0 0) 0x7f90cc003520 con 0x7f90ec0cd3e0 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 msgr2=0x7f90ec06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 0x7f90ec06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 >> v2:172.21.15.132:3300/0 conn(0x7f90ec0ce930 msgr2=0x7f90ec0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 --2- 172.21.15.105:0/3871660876 >> v2:172.21.15.132:3300/0 conn(0x7f90ec0ce930 0x7f90ec0add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec0cde80 con 0x7f90ec0cd3e0 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f90d4001670 con 0x7f90ec0cd3e0 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 >> v1:172.21.15.105:6789/0 conn(0x7f90ec0cd3e0 legacy=0x7f90ec0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 0x7f90cc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90cc003f40 0x7f90cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc004d00 0x7f90cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec06c760 con 0x7f90cc003f40 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0b42d0 con 0x7f90cc004d00 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc00b980 con 0x7f90ec06ce90 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90e3e00640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90cc003f40 0x7f90cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.671+0000 7f90ebe00640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 0x7f90cc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.672+0000 7f90f1200640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc004d00 0x7f90cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.672+0000 7f90e3e00640 1 -- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 msgr2=0x7f90cc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.672+0000 7f90e3e00640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec06ce90 0x7f90cc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.672+0000 7f90e3e00640 1 -- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc004d00 msgr2=0x7f90cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.672+0000 7f90e3e00640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc004d00 0x7f90cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.672+0000 7f90e3e00640 1 -- 172.21.15.105:0/3871660876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc003520 con 0x7f90cc003f40 2024-03-23T07:57:26.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.672+0000 7f90f1200640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc004d00 0x7f90cc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:26.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.673+0000 7f90e3e00640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90cc003f40 0x7f90cc0047c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f90d400c140 tx=0x7f90cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373ec7c1185e3cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:26.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.673+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4005ad0 con 0x7f90cc003f40 2024-03-23T07:57:26.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.673+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4005c50 con 0x7f90cc003f40 2024-03-23T07:57:26.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.673+0000 7f90eb400640 1 -- 172.21.15.105:0/3871660876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d402c810 con 0x7f90cc003f40 2024-03-23T07:57:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.674+0000 7f90f3200640 1 -- 172.21.15.105:0/3871660876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90cc003f40 msgr2=0x7f90cc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.674+0000 7f90f3200640 1 --2- 172.21.15.105:0/3871660876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90cc003f40 0x7f90cc0047c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f90d400c140 tx=0x7f90cc0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.674+0000 7f90f1200640 1 -- 172.21.15.105:0/3871660876 reap_dead start 2024-03-23T07:57:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.674+0000 7f90f3200640 1 -- 172.21.15.105:0/3871660876 shutdown_connections 2024-03-23T07:57:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.674+0000 7f90f3200640 1 -- 172.21.15.105:0/3871660876 >> 172.21.15.105:0/3871660876 conn(0x7f90ec06a0b0 msgr2=0x7f90ec06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.674+0000 7f90f3200640 1 -- 172.21.15.105:0/3871660876 shutdown_connections 2024-03-23T07:57:26.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.674+0000 7f90f3200640 1 -- 172.21.15.105:0/3871660876 wait complete. 2024-03-23T07:57:26.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.676+0000 7f90f3200640 1 Processor -- start 2024-03-23T07:57:26.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.676+0000 7f90f3200640 1 -- start start 2024-03-23T07:57:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.677+0000 7f90f3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc003f40 0x7f90ec151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.677+0000 7f90f3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec154c00 0x7f90ec1ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.677+0000 7f90f3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ec13ddb0 0x7f90ec13e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.677+0000 7f90f3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec1eff10 con 0x7f90ec13ddb0 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.677+0000 7f90f3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec1520d0 con 0x7f90cc003f40 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.677+0000 7f90f3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec153e40 con 0x7f90ec154c00 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ec13ddb0 0x7f90ec13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ec13ddb0 0x7f90ec13e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36452/0 (socket says 172.21.15.105:36452) 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 -- 172.21.15.105:0/2087848261 learned_addr learned my addr 172.21.15.105:0/2087848261 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90e3e00640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec154c00 0x7f90ec1ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90ebe00640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc003f40 0x7f90ec151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 -- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec154c00 msgr2=0x7f90ec1ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec154c00 0x7f90ec1ef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 -- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc003f40 msgr2=0x7f90ec151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc003f40 0x7f90ec151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 -- 172.21.15.105:0/2087848261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec154120 con 0x7f90ec13ddb0 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90e3e00640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec154c00 0x7f90ec1ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90ebe00640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc003f40 0x7f90ec151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.678+0000 7f90f1200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ec13ddb0 0x7f90ec13e230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90ec06fde0 tx=0x7f90e400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.679+0000 7f90e8c00640 1 -- 172.21.15.105:0/2087848261 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4017400 con 0x7f90ec13ddb0 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.679+0000 7f90e8c00640 1 -- 172.21.15.105:0/2087848261 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4011040 con 0x7f90ec13ddb0 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.679+0000 7f90e8c00640 1 -- 172.21.15.105:0/2087848261 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4017780 con 0x7f90ec13ddb0 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.679+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ec154400 con 0x7f90ec13ddb0 2024-03-23T07:57:26.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.679+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ec13e770 con 0x7f90ec13ddb0 2024-03-23T07:57:26.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.680+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ec073390 con 0x7f90ec13ddb0 2024-03-23T07:57:26.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.681+0000 7f90e8c00640 1 -- 172.21.15.105:0/2087848261 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f90e40203b0 con 0x7f90ec13ddb0 2024-03-23T07:57:26.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.682+0000 7f90e8c00640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f90c806ac00 0x7f90c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:26.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.682+0000 7f90e8c00640 1 -- 172.21.15.105:0/2087848261 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f90e4089930 con 0x7f90ec13ddb0 2024-03-23T07:57:26.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.682+0000 7f90ebe00640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f90c806ac00 0x7f90c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:26.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.683+0000 7f90ebe00640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f90c806ac00 0x7f90c806d0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009a80 tx=0x7f90dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.687+0000 7f90e8c00640 1 -- 172.21.15.105:0/2087848261 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f90e4054720 con 0x7f90ec13ddb0 2024-03-23T07:57:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.882+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90ec06c540 con 0x7f90ec13ddb0 2024-03-23T07:57:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.882+0000 7f90e8c00640 1 -- 172.21.15.105:0/2087848261 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90e4057d40 con 0x7f90ec13ddb0 2024-03-23T07:57:26.882 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f90c806ac00 msgr2=0x7f90c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f90c806ac00 0x7f90c806d0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009a80 tx=0x7f90dc008040 comp rx=0 tx=0).stop 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ec13ddb0 msgr2=0x7f90ec13e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ec13ddb0 0x7f90ec13e230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90ec06fde0 tx=0x7f90e400f4a0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 shutdown_connections 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ec13ddb0 0x7f90ec13e230 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f90c806ac00 0x7f90c806d0c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90ec154c00 0x7f90ec1ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 --2- 172.21.15.105:0/2087848261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90cc003f40 0x7f90ec151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 >> 172.21.15.105:0/2087848261 conn(0x7f90ec06a0b0 msgr2=0x7f90ec0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 shutdown_connections 2024-03-23T07:57:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:26.885+0000 7f90f3200640 1 -- 172.21.15.105:0/2087848261 wait complete. 2024-03-23T07:57:26.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.116+0000 7f062ea00640 1 Processor -- start 2024-03-23T07:57:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.116+0000 7f062ea00640 1 -- start start 2024-03-23T07:57:27.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.118+0000 7f062ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.118+0000 7f062ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0628160e90 0x7f06280cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.118+0000 7f062ea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06280cfa30 con 0x7f06280cf5f0 2024-03-23T07:57:27.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.118+0000 7f062ea00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280cdd70 con 0x7f0628160e90 2024-03-23T07:57:27.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.118+0000 7f062ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06280ccd90 con 0x7f062815c820 2024-03-23T07:57:27.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.119+0000 7f062cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0628160e90 0x7f06280cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.119+0000 7f062d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.119+0000 7f062cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0628160e90 0x7f06280cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:57626/0 (socket says 172.21.15.105:57626) 2024-03-23T07:57:27.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.119+0000 7f062cc00640 1 -- 172.21.15.105:0/2089182683 learned_addr learned my addr 172.21.15.105:0/2089182683 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:27.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.120+0000 7f062cc00640 1 -- 172.21.15.105:0/2089182683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 msgr2=0x7f06280cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:27.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.120+0000 7f062cc00640 1 --2- 172.21.15.105:0/2089182683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.120+0000 7f062cc00640 1 -- 172.21.15.105:0/2089182683 >> v1:172.21.15.132:6789/0 conn(0x7f06280cf5f0 legacy=0x7f06280d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.120+0000 7f062cc00640 1 -- 172.21.15.105:0/2089182683 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280b7fc0 con 0x7f0628160e90 2024-03-23T07:57:27.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.121+0000 7f062cc00640 1 --2- 172.21.15.105:0/2089182683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280cf0b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0614009870 tx=0x7f061402ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e75ffc5b01639d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:27.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.121+0000 7f0627e00640 1 -- 172.21.15.105:0/2089182683 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061402f830 con 0x7f0628160e90 2024-03-23T07:57:27.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f0627e00640 1 -- 172.21.15.105:0/2089182683 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061402f990 con 0x7f0628160e90 2024-03-23T07:57:27.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f0627e00640 1 -- 172.21.15.105:0/2089182683 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061402fd20 con 0x7f0628160e90 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f062ea00640 1 -- 172.21.15.105:0/2089182683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 msgr2=0x7f06280cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f062ea00640 1 --2- 172.21.15.105:0/2089182683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280cf0b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0614009870 tx=0x7f061402ba10 comp rx=0 tx=0).stop 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f062ea00640 1 -- 172.21.15.105:0/2089182683 shutdown_connections 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f062ea00640 1 --2- 172.21.15.105:0/2089182683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280cf0b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f062ea00640 1 --2- 172.21.15.105:0/2089182683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.122+0000 7f062ea00640 1 -- 172.21.15.105:0/2089182683 >> 172.21.15.105:0/2089182683 conn(0x7f062806b050 msgr2=0x7f062806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.123+0000 7f062ea00640 1 -- 172.21.15.105:0/2089182683 shutdown_connections 2024-03-23T07:57:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.123+0000 7f062ea00640 1 -- 172.21.15.105:0/2089182683 wait complete. 2024-03-23T07:57:27.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.125+0000 7f062ea00640 1 Processor -- start 2024-03-23T07:57:27.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.125+0000 7f062ea00640 1 -- start start 2024-03-23T07:57:27.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06280cf5f0 0x7f062815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062ea00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06281f1ec0 con 0x7f06280cf5f0 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062815ee50 con 0x7f0628160e90 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062815f130 con 0x7f062815c820 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062d600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06280cf5f0 0x7f062815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062d600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06280cf5f0 0x7f062815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37328/0 (socket says 172.21.15.105:37328) 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.126+0000 7f062d600640 1 -- 172.21.15.105:0/2101139101 learned_addr learned my addr 172.21.15.105:0/2101139101 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062cc00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062e000640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062d600640 1 -- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 msgr2=0x7f06280b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062d600640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062d600640 1 -- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 msgr2=0x7f06280b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062d600640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062d600640 1 -- 172.21.15.105:0/2101139101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140094d0 con 0x7f06280cf5f0 2024-03-23T07:57:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062e000640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f062d600640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06280cf5f0 0x7f062815e450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f061c00e870 tx=0x7f061c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.127+0000 7f0625600640 1 -- 172.21.15.105:0/2101139101 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0094f0 con 0x7f06280cf5f0 2024-03-23T07:57:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.128+0000 7f0625600640 1 -- 172.21.15.105:0/2101139101 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c00c940 con 0x7f06280cf5f0 2024-03-23T07:57:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.128+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062815f410 con 0x7f06280cf5f0 2024-03-23T07:57:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.128+0000 7f0625600640 1 -- 172.21.15.105:0/2101139101 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00ccf0 con 0x7f06280cf5f0 2024-03-23T07:57:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.128+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062815f9d0 con 0x7f06280cf5f0 2024-03-23T07:57:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.129+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06280cd2f0 con 0x7f06280cf5f0 2024-03-23T07:57:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.130+0000 7f0625600640 1 -- 172.21.15.105:0/2101139101 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f061c013470 con 0x7f06280cf5f0 2024-03-23T07:57:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.131+0000 7f0625600640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f060806ac00 0x7f060806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.131+0000 7f0625600640 1 -- 172.21.15.105:0/2101139101 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f061c089880 con 0x7f06280cf5f0 2024-03-23T07:57:27.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.131+0000 7f062cc00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f060806ac00 0x7f060806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.132+0000 7f062cc00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f060806ac00 0x7f060806d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0614036000 tx=0x7f0614008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:27.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.135+0000 7f0625600640 1 -- 172.21.15.105:0/2101139101 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f06280cd2f0 con 0x7f06280cf5f0 2024-03-23T07:57:27.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.332+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06280cd2f0 con 0x7f06280cf5f0 2024-03-23T07:57:27.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.333+0000 7f0625600640 1 -- 172.21.15.105:0/2101139101 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06280cd2f0 con 0x7f06280cf5f0 2024-03-23T07:57:27.333 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:27.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.335+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f060806ac00 msgr2=0x7f060806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.335+0000 7f062ea00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f060806ac00 0x7f060806d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0614036000 tx=0x7f0614008030 comp rx=0 tx=0).stop 2024-03-23T07:57:27.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.335+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06280cf5f0 msgr2=0x7f062815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.335+0000 7f062ea00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06280cf5f0 0x7f062815e450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f061c00e870 tx=0x7f061c00ed30 comp rx=0 tx=0).stop 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 shutdown_connections 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0628160e90 0x7f06280b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f060806ac00 0x7f060806d0c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f062815c820 0x7f06280b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 --2- 172.21.15.105:0/2101139101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06280cf5f0 0x7f062815e450 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 >> 172.21.15.105:0/2101139101 conn(0x7f062806b050 msgr2=0x7f06280c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 shutdown_connections 2024-03-23T07:57:27.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.336+0000 7f062ea00640 1 -- 172.21.15.105:0/2101139101 wait complete. 2024-03-23T07:57:27.355 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.567+0000 7f6e23a00640 1 Processor -- start 2024-03-23T07:57:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.567+0000 7f6e23a00640 1 -- start start 2024-03-23T07:57:27.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e23a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c0ca7b0 0x7f6e1c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e23a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e1c0ce930 0x7f6e1c0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e23a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1c159c10 con 0x7f6e1c159830 2024-03-23T07:57:27.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e23a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c06de50 con 0x7f6e1c0ce930 2024-03-23T07:57:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e23a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c0cf300 con 0x7f6e1c0ca7b0 2024-03-23T07:57:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e21a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e1c0ce930 0x7f6e1c0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e1be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c0ca7b0 0x7f6e1c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.569+0000 7f6e21a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e1c0ce930 0x7f6e1c0cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:57650/0 (socket says 172.21.15.105:57650) 2024-03-23T07:57:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.570+0000 7f6e1be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c0ca7b0 0x7f6e1c0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:56386/0 (socket says 172.21.15.105:56386) 2024-03-23T07:57:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.570+0000 7f6e21a00640 1 -- 172.21.15.105:0/3651550259 learned_addr learned my addr 172.21.15.105:0/3651550259 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.570+0000 7f6e1be00640 1 -- 172.21.15.105:0/3651550259 >> v2:172.21.15.132:3300/0 conn(0x7f6e1c0ce930 msgr2=0x7f6e1c0cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.570+0000 7f6e1be00640 1 --2- 172.21.15.105:0/3651550259 >> v2:172.21.15.132:3300/0 conn(0x7f6e1c0ce930 0x7f6e1c0cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.570+0000 7f6e1be00640 1 -- 172.21.15.105:0/3651550259 >> v1:172.21.15.132:6789/0 conn(0x7f6e1c159830 legacy=0x7f6e1c0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.570+0000 7f6e1be00640 1 -- 172.21.15.105:0/3651550259 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c0cd840 con 0x7f6e1c0ca7b0 2024-03-23T07:57:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.573+0000 7f6e1be00640 1 --2- 172.21.15.105:0/3651550259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c0ca7b0 0x7f6e1c0cdcd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800ba40 tx=0x7f6e0802e110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f00ae484f874ca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.573+0000 7f6e1b400640 1 -- 172.21.15.105:0/3651550259 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08036440 con 0x7f6e1c0ca7b0 2024-03-23T07:57:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e1b400640 1 -- 172.21.15.105:0/3651550259 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e080365a0 con 0x7f6e1c0ca7b0 2024-03-23T07:57:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e1b400640 1 -- 172.21.15.105:0/3651550259 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08036930 con 0x7f6e1c0ca7b0 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 -- 172.21.15.105:0/3651550259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c0ca7b0 msgr2=0x7f6e1c0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 --2- 172.21.15.105:0/3651550259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c0ca7b0 0x7f6e1c0cdcd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800ba40 tx=0x7f6e0802e110 comp rx=0 tx=0).stop 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 -- 172.21.15.105:0/3651550259 shutdown_connections 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 --2- 172.21.15.105:0/3651550259 >> v2:172.21.15.132:3300/0 conn(0x7f6e1c0ce930 0x7f6e1c0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 --2- 172.21.15.105:0/3651550259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c0ca7b0 0x7f6e1c0cdcd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 -- 172.21.15.105:0/3651550259 >> 172.21.15.105:0/3651550259 conn(0x7f6e1c06a0b0 msgr2=0x7f6e1c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 -- 172.21.15.105:0/3651550259 shutdown_connections 2024-03-23T07:57:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.574+0000 7f6e23a00640 1 -- 172.21.15.105:0/3651550259 wait complete. 2024-03-23T07:57:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.576+0000 7f6e23a00640 1 Processor -- start 2024-03-23T07:57:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.576+0000 7f6e23a00640 1 -- start start 2024-03-23T07:57:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.577+0000 7f6e23a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c0ca7b0 0x7f6e1c14b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.577+0000 7f6e23a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e1c0ce930 0x7f6e1c1f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.577+0000 7f6e23a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c159830 0x7f6e1c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.577+0000 7f6e23a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c15b530 con 0x7f6e1c0ce930 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.577+0000 7f6e23a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c1f2fe0 con 0x7f6e1c0ca7b0 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.577+0000 7f6e23a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c15a500 con 0x7f6e1c159830 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e1c0ce930 0x7f6e1c1f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e1c0ce930 0x7f6e1c1f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37342/0 (socket says 172.21.15.105:37342) 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 -- 172.21.15.105:0/4119323440 learned_addr learned my addr 172.21.15.105:0/4119323440 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e21000640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c0ca7b0 0x7f6e1c14b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 -- 172.21.15.105:0/4119323440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c159830 msgr2=0x7f6e1c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c159830 0x7f6e1c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 -- 172.21.15.105:0/4119323440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c0ca7b0 msgr2=0x7f6e1c14b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c0ca7b0 0x7f6e1c14b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 -- 172.21.15.105:0/4119323440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0800b6a0 con 0x7f6e1c0ce930 2024-03-23T07:57:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.578+0000 7f6e1be00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e1c0ce930 0x7f6e1c1f2aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6e08039040 tx=0x7f6e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.579+0000 7f6e18c00640 1 -- 172.21.15.105:0/4119323440 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08034b00 con 0x7f6e1c0ce930 2024-03-23T07:57:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.579+0000 7f6e18c00640 1 -- 172.21.15.105:0/4119323440 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08036e90 con 0x7f6e1c0ce930 2024-03-23T07:57:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.579+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c15ae90 con 0x7f6e1c0ce930 2024-03-23T07:57:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.579+0000 7f6e18c00640 1 -- 172.21.15.105:0/4119323440 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08006e10 con 0x7f6e1c0ce930 2024-03-23T07:57:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.579+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c15aa80 con 0x7f6e1c0ce930 2024-03-23T07:57:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.580+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c15b670 con 0x7f6e1c0ce930 2024-03-23T07:57:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.581+0000 7f6e18c00640 1 -- 172.21.15.105:0/4119323440 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6e0803d070 con 0x7f6e1c0ce930 2024-03-23T07:57:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.582+0000 7f6e18c00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6dfc06ac00 0x7f6dfc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.582+0000 7f6e18c00640 1 -- 172.21.15.105:0/4119323440 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6e080ab960 con 0x7f6e1c0ce930 2024-03-23T07:57:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.582+0000 7f6e21000640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6dfc06ac00 0x7f6dfc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:27.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.583+0000 7f6e21000640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6dfc06ac00 0x7f6dfc06d0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e10000f60 tx=0x7f6e10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:27.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.586+0000 7f6e18c00640 1 -- 172.21.15.105:0/4119323440 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6e1c15b670 con 0x7f6e1c0ce930 2024-03-23T07:57:27.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.779+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e1c15b670 con 0x7f6e1c0ce930 2024-03-23T07:57:27.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.780+0000 7f6e18c00640 1 -- 172.21.15.105:0/4119323440 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e1c15b670 con 0x7f6e1c0ce930 2024-03-23T07:57:27.780 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6dfc06ac00 msgr2=0x7f6dfc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6dfc06ac00 0x7f6dfc06d0c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e10000f60 tx=0x7f6e10009040 comp rx=0 tx=0).stop 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e1c0ce930 msgr2=0x7f6e1c1f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e1c0ce930 0x7f6e1c1f2aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6e08039040 tx=0x7f6e080046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 shutdown_connections 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6dfc06ac00 0x7f6dfc06d0c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e1c159830 0x7f6e1c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e1c0ce930 0x7f6e1c1f2aa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 --2- 172.21.15.105:0/4119323440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c0ca7b0 0x7f6e1c14b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 >> 172.21.15.105:0/4119323440 conn(0x7f6e1c06a0b0 msgr2=0x7f6e1c070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 shutdown_connections 2024-03-23T07:57:27.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:27.783+0000 7f6e23a00640 1 -- 172.21.15.105:0/4119323440 wait complete. 2024-03-23T07:57:27.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:28.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.015+0000 7f923b800640 1 Processor -- start 2024-03-23T07:57:28.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.016+0000 7f923b800640 1 -- start start 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f923b800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92340cd740 0x7f92340b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f923b800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92340d0d40 0x7f92340cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f923b800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92340d20c0 con 0x7f923406b0a0 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f923b800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92340ce060 con 0x7f92340d0d40 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f923b800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92340d14b0 con 0x7f92340cd740 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f9233e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92340d0d40 0x7f92340cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f9233e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92340d0d40 0x7f92340cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37352/0 (socket says 172.21.15.105:37352) 2024-03-23T07:57:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.017+0000 7f9233e00640 1 -- 172.21.15.105:0/1898982676 learned_addr learned my addr 172.21.15.105:0/1898982676 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.018+0000 7f9238e00640 1 --2- 172.21.15.105:0/1898982676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92340cd740 0x7f92340b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.018+0000 7f9233e00640 1 -- 172.21.15.105:0/1898982676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92340cd740 msgr2=0x7f92340b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.018+0000 7f9233e00640 1 --2- 172.21.15.105:0/1898982676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92340cd740 0x7f92340b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.018+0000 7f9233e00640 1 -- 172.21.15.105:0/1898982676 >> v1:172.21.15.105:6789/0 conn(0x7f923406b0a0 legacy=0x7f923406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.018+0000 7f9233e00640 1 -- 172.21.15.105:0/1898982676 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92340d1790 con 0x7f92340d0d40 2024-03-23T07:57:28.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.018+0000 7f9233400640 1 -- 172.21.15.105:0/1898982676 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811851108 0 0) 0x7f92340d20c0 con 0x7f923406b0a0 2024-03-23T07:57:28.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.019+0000 7f9233e00640 1 --2- 172.21.15.105:0/1898982676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340cdb20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa170810f405b82 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.019+0000 7f9233400640 1 -- 172.21.15.105:0/1898982676 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c030840 con 0x7f92340d0d40 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.019+0000 7f9233400640 1 -- 172.21.15.105:0/1898982676 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0309a0 con 0x7f92340d0d40 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.019+0000 7f9233400640 1 -- 172.21.15.105:0/1898982676 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c030d30 con 0x7f92340d0d40 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 -- 172.21.15.105:0/1898982676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 msgr2=0x7f92340cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 --2- 172.21.15.105:0/1898982676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340cdb20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c02b980 comp rx=0 tx=0).stop 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 -- 172.21.15.105:0/1898982676 shutdown_connections 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 --2- 172.21.15.105:0/1898982676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340cdb20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 --2- 172.21.15.105:0/1898982676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92340cd740 0x7f92340b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 -- 172.21.15.105:0/1898982676 >> 172.21.15.105:0/1898982676 conn(0x7f92340cf5d0 msgr2=0x7f92340cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 -- 172.21.15.105:0/1898982676 shutdown_connections 2024-03-23T07:57:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.020+0000 7f923b800640 1 -- 172.21.15.105:0/1898982676 wait complete. 2024-03-23T07:57:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.022+0000 7f923b800640 1 Processor -- start 2024-03-23T07:57:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.022+0000 7f923b800640 1 -- start start 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f923b800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f923406b0a0 0x7f92340a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f923b800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92340cd740 0x7f92340cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f923b800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f923b800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92340ca6b0 con 0x7f92340d0d40 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f923b800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92340c9ae0 con 0x7f92340cd740 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f923b800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92340c9d80 con 0x7f923406b0a0 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f9239800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f9239800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37354/0 (socket says 172.21.15.105:37354) 2024-03-23T07:57:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f9239800640 1 -- 172.21.15.105:0/2117764773 learned_addr learned my addr 172.21.15.105:0/2117764773 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.023+0000 7f9239800640 1 -- 172.21.15.105:0/2117764773 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f923406b0a0 msgr2=0x7f92340a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9239800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f923406b0a0 0x7f92340a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9239800640 1 -- 172.21.15.105:0/2117764773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92340cd740 msgr2=0x7f92340cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9239800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92340cd740 0x7f92340cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9239800640 1 -- 172.21.15.105:0/2117764773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0094d0 con 0x7f92340d0d40 2024-03-23T07:57:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9239800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340b3ca0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c760 tx=0x7f922c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:28.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9230c00640 1 -- 172.21.15.105:0/2117764773 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c003da0 con 0x7f92340d0d40 2024-03-23T07:57:28.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9230c00640 1 -- 172.21.15.105:0/2117764773 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922c00ce70 con 0x7f92340d0d40 2024-03-23T07:57:28.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92340ca060 con 0x7f92340d0d40 2024-03-23T07:57:28.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f9230c00640 1 -- 172.21.15.105:0/2117764773 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c005290 con 0x7f92340d0d40 2024-03-23T07:57:28.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.024+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92340a97b0 con 0x7f92340d0d40 2024-03-23T07:57:28.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.026+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9234073390 con 0x7f92340d0d40 2024-03-23T07:57:28.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.026+0000 7f9230c00640 1 -- 172.21.15.105:0/2117764773 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f922c0053f0 con 0x7f92340d0d40 2024-03-23T07:57:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.027+0000 7f9230c00640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f921006ac00 0x7f921006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.027+0000 7f9230c00640 1 -- 172.21.15.105:0/2117764773 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f922c089590 con 0x7f92340d0d40 2024-03-23T07:57:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.028+0000 7f9238e00640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f921006ac00 0x7f921006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.028+0000 7f9238e00640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f921006ac00 0x7f921006d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92240098f0 tx=0x7f9224008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.033+0000 7f9230c00640 1 -- 172.21.15.105:0/2117764773 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f922c054420 con 0x7f92340d0d40 2024-03-23T07:57:28.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.232+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f92340ca7f0 con 0x7f92340d0d40 2024-03-23T07:57:28.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.232+0000 7f9230c00640 1 -- 172.21.15.105:0/2117764773 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f922c057a40 con 0x7f92340d0d40 2024-03-23T07:57:28.232 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f921006ac00 msgr2=0x7f921006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f921006ac00 0x7f921006d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92240098f0 tx=0x7f9224008040 comp rx=0 tx=0).stop 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 msgr2=0x7f92340b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340b3ca0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c760 tx=0x7f922c00cc20 comp rx=0 tx=0).stop 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 shutdown_connections 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92340d0d40 0x7f92340b3ca0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92340cd740 0x7f92340cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f921006ac00 0x7f921006d0c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 --2- 172.21.15.105:0/2117764773 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f923406b0a0 0x7f92340a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 >> 172.21.15.105:0/2117764773 conn(0x7f92340cf5d0 msgr2=0x7f92340b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 shutdown_connections 2024-03-23T07:57:28.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.235+0000 7f923b800640 1 -- 172.21.15.105:0/2117764773 wait complete. 2024-03-23T07:57:28.255 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T07:57:28.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.467+0000 7ff7b3a00640 1 Processor -- start 2024-03-23T07:57:28.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.467+0000 7ff7b3a00640 1 -- start start 2024-03-23T07:57:28.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.468+0000 7ff7b3a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7ac0ce930 0x7ff7ac0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.468+0000 7ff7b3a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac070f90 con 0x7ff7ac0cb660 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.468+0000 7ff7b3a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac0cf250 con 0x7ff7ac06d2c0 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.468+0000 7ff7b3a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac15bae0 con 0x7ff7ac0ce930 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.469+0000 7ff7b1000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7ac0ce930 0x7ff7ac0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.469+0000 7ff7abe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff7ac0cb660 0x7ff7ac070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60476/0 (socket says 172.21.15.105:60476) 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.469+0000 7ff7abe00640 1 -- 172.21.15.105:0/1277611344 learned_addr learned my addr 172.21.15.105:0/1277611344 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.469+0000 7ff7b1000640 1 -- 172.21.15.105:0/1277611344 >> v1:172.21.15.132:6789/0 conn(0x7ff7ac06d2c0 legacy=0x7ff7ac070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.469+0000 7ff7b1000640 1 -- 172.21.15.105:0/1277611344 >> v1:172.21.15.105:6789/0 conn(0x7ff7ac0cb660 legacy=0x7ff7ac070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.469+0000 7ff7b1000640 1 -- 172.21.15.105:0/1277611344 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac15ae60 con 0x7ff7ac0ce930 2024-03-23T07:57:28.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.470+0000 7ff7b1000640 1 --2- 172.21.15.105:0/1277611344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac0ced10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00097a0 tx=0x7ff7a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ee0a061df3b19e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7ab400640 1 -- 172.21.15.105:0/1277611344 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0031830 con 0x7ff7ac0ce930 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7ab400640 1 -- 172.21.15.105:0/1277611344 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a0031990 con 0x7ff7ac0ce930 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7ab400640 1 -- 172.21.15.105:0/1277611344 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0031d20 con 0x7ff7ac0ce930 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1277611344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 msgr2=0x7ff7ac0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1277611344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac0ced10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00097a0 tx=0x7ff7a002b8d0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1277611344 shutdown_connections 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1277611344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac0ced10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1277611344 >> 172.21.15.105:0/1277611344 conn(0x7ff7ac06a0b0 msgr2=0x7ff7ac06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.471+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1277611344 shutdown_connections 2024-03-23T07:57:28.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.472+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1277611344 wait complete. 2024-03-23T07:57:28.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.473+0000 7ff7b3a00640 1 Processor -- start 2024-03-23T07:57:28.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.473+0000 7ff7b3a00640 1 -- start start 2024-03-23T07:57:28.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.474+0000 7ff7b3a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7ac06d2c0 0x7ff7ac1f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.474+0000 7ff7b3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7ac0cb660 0x7ff7ac14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.474+0000 7ff7b3a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.474+0000 7ff7b3a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac14b7f0 con 0x7ff7ac0ce930 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.474+0000 7ff7b3a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac156930 con 0x7ff7ac06d2c0 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.474+0000 7ff7b3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac156bb0 con 0x7ff7ac0cb660 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac1563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37376/0 (socket says 172.21.15.105:37376) 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 -- 172.21.15.105:0/1384428865 learned_addr learned my addr 172.21.15.105:0/1384428865 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1000640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7ac06d2c0 0x7ff7ac1f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 -- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7ac0cb660 msgr2=0x7ff7ac14b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7abe00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7ac0cb660 0x7ff7ac14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7ac0cb660 0x7ff7ac14b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 -- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7ac06d2c0 msgr2=0x7ff7ac1f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7ac06d2c0 0x7ff7ac1f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 -- 172.21.15.105:0/1384428865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a0009400 con 0x7ff7ac0ce930 2024-03-23T07:57:28.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.475+0000 7ff7b1a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac1563f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c780 tx=0x7ff7a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:28.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.476+0000 7ff7a8c00640 1 -- 172.21.15.105:0/1384428865 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4003d80 con 0x7ff7ac0ce930 2024-03-23T07:57:28.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.476+0000 7ff7a8c00640 1 -- 172.21.15.105:0/1384428865 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a400ce90 con 0x7ff7ac0ce930 2024-03-23T07:57:28.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.476+0000 7ff7a8c00640 1 -- 172.21.15.105:0/1384428865 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40052b0 con 0x7ff7ac0ce930 2024-03-23T07:57:28.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.476+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac156e90 con 0x7ff7ac0ce930 2024-03-23T07:57:28.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.476+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac157450 con 0x7ff7ac0ce930 2024-03-23T07:57:28.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.477+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac073390 con 0x7ff7ac0ce930 2024-03-23T07:57:28.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.478+0000 7ff7a8c00640 1 -- 172.21.15.105:0/1384428865 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff7a4005410 con 0x7ff7ac0ce930 2024-03-23T07:57:28.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.479+0000 7ff7a8c00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff79006ac30 0x7ff79006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:28.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.479+0000 7ff7a8c00640 1 -- 172.21.15.105:0/1384428865 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff7a4088ae0 con 0x7ff7ac0ce930 2024-03-23T07:57:28.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.479+0000 7ff7b1000640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff79006ac30 0x7ff79006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:28.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.480+0000 7ff7b1000640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff79006ac30 0x7ff79006d0f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0036000 tx=0x7ff7a003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:28.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.483+0000 7ff7a8c00640 1 -- 172.21.15.105:0/1384428865 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff7a40538d0 con 0x7ff7ac0ce930 2024-03-23T07:57:28.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.689+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff7ac06a530 con 0x7ff7ac0ce930 2024-03-23T07:57:28.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.690+0000 7ff7a8c00640 1 -- 172.21.15.105:0/1384428865 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff7a4017080 con 0x7ff7ac0ce930 2024-03-23T07:57:28.689 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-23T07:57:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff79006ac30 msgr2=0x7ff79006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff79006ac30 0x7ff79006d0f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0036000 tx=0x7ff7a003c040 comp rx=0 tx=0).stop 2024-03-23T07:57:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 msgr2=0x7ff7ac1563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac1563f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c780 tx=0x7ff7a400cc40 comp rx=0 tx=0).stop 2024-03-23T07:57:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 shutdown_connections 2024-03-23T07:57:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7ac0ce930 0x7ff7ac1563f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff79006ac30 0x7ff79006d0f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7ac0cb660 0x7ff7ac14b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.692+0000 7ff7b3a00640 1 --2- 172.21.15.105:0/1384428865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7ac06d2c0 0x7ff7ac1f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.693+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 >> 172.21.15.105:0/1384428865 conn(0x7ff7ac06a0b0 msgr2=0x7ff7ac158970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.693+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 shutdown_connections 2024-03-23T07:57:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:28.693+0000 7ff7b3a00640 1 -- 172.21.15.105:0/1384428865 wait complete. 2024-03-23T07:57:28.710 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-23T07:57:28.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:28.710 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-23T07:57:28.818 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:57:28.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:28.818 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-23T07:57:28.887 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:28.887 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:28.941 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:28.942 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:28.992 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:57:28.992 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:29.041 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:29.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.301+0000 7fec83200640 1 Processor -- start 2024-03-23T07:57:29.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.301+0000 7fec83200640 1 -- start start 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.302+0000 7fec83200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec7c15b560 0x7fec7c15b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.302+0000 7fec83200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c15d410 0x7fec7c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.302+0000 7fec83200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c06c6f0 con 0x7fec7c0b39b0 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.302+0000 7fec83200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c0b4370 con 0x7fec7c15b560 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.302+0000 7fec83200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c06c9d0 con 0x7fec7c15d410 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.303+0000 7fec7be00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec7c15b560 0x7fec7c15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.303+0000 7fec7be00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec7c15b560 0x7fec7c15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37384/0 (socket says 172.21.15.105:37384) 2024-03-23T07:57:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.303+0000 7fec7be00640 1 -- 172.21.15.105:0/1731930534 learned_addr learned my addr 172.21.15.105:0/1731930534 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:29.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.303+0000 7fec7be00640 1 -- 172.21.15.105:0/1731930534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c15d410 msgr2=0x7fec7c0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:29.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.303+0000 7fec7be00640 1 --2- 172.21.15.105:0/1731930534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c15d410 0x7fec7c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.303+0000 7fec7be00640 1 -- 172.21.15.105:0/1731930534 >> v1:172.21.15.132:6789/0 conn(0x7fec7c0b39b0 legacy=0x7fec7c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:29.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.303+0000 7fec7be00640 1 -- 172.21.15.105:0/1731930534 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c06ccb0 con 0x7fec7c15b560 2024-03-23T07:57:29.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.304+0000 7fec7be00640 1 --2- 172.21.15.105:0/1731930534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15b560 0x7fec7c15b940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009870 tx=0x7fec6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e37d2948d0de26 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:29.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.304+0000 7fec7b400640 1 -- 172.21.15.105:0/1731930534 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c031640 con 0x7fec7c15b560 2024-03-23T07:57:29.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec7b400640 1 -- 172.21.15.105:0/1731930534 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c0317a0 con 0x7fec7c15b560 2024-03-23T07:57:29.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec7b400640 1 -- 172.21.15.105:0/1731930534 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c031b30 con 0x7fec7c15b560 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 -- 172.21.15.105:0/1731930534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15b560 msgr2=0x7fec7c15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 --2- 172.21.15.105:0/1731930534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15b560 0x7fec7c15b940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009870 tx=0x7fec6c02b8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 -- 172.21.15.105:0/1731930534 shutdown_connections 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 --2- 172.21.15.105:0/1731930534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c15d410 0x7fec7c0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 --2- 172.21.15.105:0/1731930534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15b560 0x7fec7c15b940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 -- 172.21.15.105:0/1731930534 >> 172.21.15.105:0/1731930534 conn(0x7fec7c06a3d0 msgr2=0x7fec7c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 -- 172.21.15.105:0/1731930534 shutdown_connections 2024-03-23T07:57:29.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.305+0000 7fec83200640 1 -- 172.21.15.105:0/1731930534 wait complete. 2024-03-23T07:57:29.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.307+0000 7fec83200640 1 Processor -- start 2024-03-23T07:57:29.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.307+0000 7fec83200640 1 -- start start 2024-03-23T07:57:29.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec83200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c0b39b0 0x7fec7c1efc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec83200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec7c15b560 0x7fec7c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec83200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15d410 0x7fec7c155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec83200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c157a60 con 0x7fec7c15d410 2024-03-23T07:57:29.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec83200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c1f01d0 con 0x7fec7c15b560 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec83200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c13f7d0 con 0x7fec7c0b39b0 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec81200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15d410 0x7fec7c155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec81200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15d410 0x7fec7c155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37392/0 (socket says 172.21.15.105:37392) 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.308+0000 7fec81200640 1 -- 172.21.15.105:0/1778988602 learned_addr learned my addr 172.21.15.105:0/1778988602 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec7be00640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c0b39b0 0x7fec7c1efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec73400640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec7c15b560 0x7fec7c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec81200640 1 -- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c0b39b0 msgr2=0x7fec7c1efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec81200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c0b39b0 0x7fec7c1efc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec81200640 1 -- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec7c15b560 msgr2=0x7fec7c155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec81200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec7c15b560 0x7fec7c155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec81200640 1 -- 172.21.15.105:0/1778988602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0094d0 con 0x7fec7c15d410 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec73400640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec7c15b560 0x7fec7c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:29.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec81200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15d410 0x7fec7c155a60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec7400b970 tx=0x7fec7400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:29.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec78c00640 1 -- 172.21.15.105:0/1778988602 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400cb40 con 0x7fec7c15d410 2024-03-23T07:57:29.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec78c00640 1 -- 172.21.15.105:0/1778988602 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400ccc0 con 0x7fec7c15d410 2024-03-23T07:57:29.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec78c00640 1 -- 172.21.15.105:0/1778988602 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74014680 con 0x7fec7c15d410 2024-03-23T07:57:29.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.309+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c13fab0 con 0x7fec7c15d410 2024-03-23T07:57:29.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.310+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c12d180 con 0x7fec7c15d410 2024-03-23T07:57:29.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.311+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c12cd00 con 0x7fec7c15d410 2024-03-23T07:57:29.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.312+0000 7fec78c00640 1 -- 172.21.15.105:0/1778988602 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fec7401d3b0 con 0x7fec7c15d410 2024-03-23T07:57:29.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.312+0000 7fec78c00640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec5806ac00 0x7fec5806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.312+0000 7fec78c00640 1 -- 172.21.15.105:0/1778988602 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fec74089720 con 0x7fec7c15d410 2024-03-23T07:57:29.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.313+0000 7fec7be00640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec5806ac00 0x7fec5806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.313+0000 7fec7be00640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec5806ac00 0x7fec5806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec6c036000 tx=0x7fec6c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:29.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.317+0000 7fec78c00640 1 -- 172.21.15.105:0/1778988602 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fec7c12cd00 con 0x7fec7c15d410 2024-03-23T07:57:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.509+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec7c12cd00 con 0x7fec7c15d410 2024-03-23T07:57:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.511+0000 7fec78c00640 1 -- 172.21.15.105:0/1778988602 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7fec7c12cd00 con 0x7fec7c15d410 2024-03-23T07:57:29.511 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:29.512 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec5806ac00 msgr2=0x7fec5806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec5806ac00 0x7fec5806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec6c036000 tx=0x7fec6c03c040 comp rx=0 tx=0).stop 2024-03-23T07:57:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15d410 msgr2=0x7fec7c155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15d410 0x7fec7c155a60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec7400b970 tx=0x7fec7400be30 comp rx=0 tx=0).stop 2024-03-23T07:57:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 shutdown_connections 2024-03-23T07:57:29.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7c15d410 0x7fec7c155a60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec7c15b560 0x7fec7c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec5806ac00 0x7fec5806d0c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 --2- 172.21.15.105:0/1778988602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec7c0b39b0 0x7fec7c1efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.514+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 >> 172.21.15.105:0/1778988602 conn(0x7fec7c06a3d0 msgr2=0x7fec7c15daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:29.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.515+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 shutdown_connections 2024-03-23T07:57:29.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.515+0000 7fec83200640 1 -- 172.21.15.105:0/1778988602 wait complete. 2024-03-23T07:57:29.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:29.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:29.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.744+0000 7f80fcc00640 1 Processor -- start 2024-03-23T07:57:29.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.745+0000 7f80fcc00640 1 -- start start 2024-03-23T07:57:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.746+0000 7f80fcc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f815cc30 0x7f80f80d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.746+0000 7f80fcc00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80f80d1be0 0x7f80f80b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.746+0000 7f80fcc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f80f80b60d0 0x7f80f80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.746+0000 7f80fcc00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f80b6560 con 0x7f80f80d1be0 2024-03-23T07:57:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.746+0000 7f80fcc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f8160ae0 con 0x7f80f80b60d0 2024-03-23T07:57:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.746+0000 7f80fcc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80d0c00 con 0x7f80f815cc30 2024-03-23T07:57:29.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80f80d1be0 0x7f80f80b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80f80d1be0 0x7f80f80b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37404/0 (socket says 172.21.15.105:37404) 2024-03-23T07:57:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 -- 172.21.15.105:0/1750161975 learned_addr learned my addr 172.21.15.105:0/1750161975 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f7e00640 1 --2- 172.21.15.105:0/1750161975 >> v2:172.21.15.132:3300/0 conn(0x7f80f80b60d0 0x7f80f80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f7400640 1 --2- 172.21.15.105:0/1750161975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f815cc30 0x7f80f80d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 -- 172.21.15.105:0/1750161975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f815cc30 msgr2=0x7f80f80d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 --2- 172.21.15.105:0/1750161975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f815cc30 0x7f80f80d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 -- 172.21.15.105:0/1750161975 >> v2:172.21.15.132:3300/0 conn(0x7f80f80b60d0 msgr2=0x7f80f80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 --2- 172.21.15.105:0/1750161975 >> v2:172.21.15.132:3300/0 conn(0x7f80f80b60d0 0x7f80f80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f6a00640 1 -- 172.21.15.105:0/1750161975 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f80cdde0 con 0x7f80f80d1be0 2024-03-23T07:57:29.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.747+0000 7f80f7e00640 1 --2- 172.21.15.105:0/1750161975 >> v2:172.21.15.132:3300/0 conn(0x7f80f80b60d0 0x7f80f80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.748+0000 7f80f6a00640 1 --2- 172.21.15.105:0/1750161975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f80b5b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f80f806bf60 tx=0x7f80e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f74849e17b7de6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80f6000640 1 -- 172.21.15.105:0/1750161975 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4031640 con 0x7f80f80d1be0 2024-03-23T07:57:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80f6000640 1 -- 172.21.15.105:0/1750161975 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e40317a0 con 0x7f80f80d1be0 2024-03-23T07:57:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80f6000640 1 -- 172.21.15.105:0/1750161975 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4031b30 con 0x7f80f80d1be0 2024-03-23T07:57:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 -- 172.21.15.105:0/1750161975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 msgr2=0x7f80f80b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 --2- 172.21.15.105:0/1750161975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f80b5b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f80f806bf60 tx=0x7f80e402b8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 -- 172.21.15.105:0/1750161975 shutdown_connections 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 --2- 172.21.15.105:0/1750161975 >> v2:172.21.15.132:3300/0 conn(0x7f80f80b60d0 0x7f80f80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 --2- 172.21.15.105:0/1750161975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f80b5b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 --2- 172.21.15.105:0/1750161975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f815cc30 0x7f80f80d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 -- 172.21.15.105:0/1750161975 >> 172.21.15.105:0/1750161975 conn(0x7f80f806b100 msgr2=0x7f80f806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 -- 172.21.15.105:0/1750161975 shutdown_connections 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.749+0000 7f80fcc00640 1 -- 172.21.15.105:0/1750161975 wait complete. 2024-03-23T07:57:29.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.751+0000 7f80fcc00640 1 Processor -- start 2024-03-23T07:57:29.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.751+0000 7f80fcc00640 1 -- start start 2024-03-23T07:57:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80fcc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f80b60d0 0x7f80f81f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80fcc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80fcc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f815cc30 0x7f80f815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80fcc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80b1e50 con 0x7f80f80d1be0 2024-03-23T07:57:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80fcc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f8157070 con 0x7f80f815cc30 2024-03-23T07:57:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80fcc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f8157350 con 0x7f80f80b60d0 2024-03-23T07:57:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80f6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80f6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f815f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37416/0 (socket says 172.21.15.105:37416) 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.752+0000 7f80f6a00640 1 -- 172.21.15.105:0/3654399017 learned_addr learned my addr 172.21.15.105:0/3654399017 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80f6a00640 1 -- 172.21.15.105:0/3654399017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f80b60d0 msgr2=0x7f80f81f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80f7e00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f815cc30 0x7f80f815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80f6a00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f80b60d0 0x7f80f81f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80f6a00640 1 -- 172.21.15.105:0/3654399017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f815cc30 msgr2=0x7f80f815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80f6a00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f815cc30 0x7f80f815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80f6a00640 1 -- 172.21.15.105:0/3654399017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40094d0 con 0x7f80f80d1be0 2024-03-23T07:57:29.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80f6a00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f815f360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:29.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80ef400640 1 -- 172.21.15.105:0/3654399017 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e403ac60 con 0x7f80f80d1be0 2024-03-23T07:57:29.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.753+0000 7f80ef400640 1 -- 172.21.15.105:0/3654399017 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e403ade0 con 0x7f80f80d1be0 2024-03-23T07:57:29.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.754+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f80b20f0 con 0x7f80f80d1be0 2024-03-23T07:57:29.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.754+0000 7f80ef400640 1 -- 172.21.15.105:0/3654399017 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4044630 con 0x7f80f80d1be0 2024-03-23T07:57:29.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.754+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f81f1e90 con 0x7f80f80d1be0 2024-03-23T07:57:29.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.755+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f804e030 con 0x7f80f80d1be0 2024-03-23T07:57:29.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.756+0000 7f80ef400640 1 -- 172.21.15.105:0/3654399017 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f80e402f050 con 0x7f80f80d1be0 2024-03-23T07:57:29.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.757+0000 7f80ef400640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80d806ac00 0x7f80d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:29.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.757+0000 7f80ef400640 1 -- 172.21.15.105:0/3654399017 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f80e40ab800 con 0x7f80f80d1be0 2024-03-23T07:57:29.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.757+0000 7f80f7400640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80d806ac00 0x7f80d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:29.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.758+0000 7f80f7400640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80d806ac00 0x7f80d806d0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f80e800a810 tx=0x7f80e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:29.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.762+0000 7f80ef400640 1 -- 172.21.15.105:0/3654399017 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f80e4076680 con 0x7f80f80d1be0 2024-03-23T07:57:29.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.953+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80f80d0710 con 0x7f80f80d1be0 2024-03-23T07:57:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.954+0000 7f80ef400640 1 -- 172.21.15.105:0/3654399017 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80e4079ca0 con 0x7f80f80d1be0 2024-03-23T07:57:29.954 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.956+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80d806ac00 msgr2=0x7f80d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.956+0000 7f80fcc00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80d806ac00 0x7f80d806d0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f80e800a810 tx=0x7f80e8008040 comp rx=0 tx=0).stop 2024-03-23T07:57:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.956+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 msgr2=0x7f80f815f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.956+0000 7f80fcc00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f815f360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e40046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.956+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 shutdown_connections 2024-03-23T07:57:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.957+0000 7f80fcc00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f815cc30 0x7f80f815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.957+0000 7f80fcc00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f80d1be0 0x7f80f815f360 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.957+0000 7f80fcc00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f80d806ac00 0x7f80d806d0c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.957+0000 7f80fcc00640 1 --2- 172.21.15.105:0/3654399017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f80b60d0 0x7f80f81f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.957+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 >> 172.21.15.105:0/3654399017 conn(0x7f80f806b100 msgr2=0x7f80f80c6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.957+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 shutdown_connections 2024-03-23T07:57:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:29.957+0000 7f80fcc00640 1 -- 172.21.15.105:0/3654399017 wait complete. 2024-03-23T07:57:29.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.184+0000 7f5a18800640 1 Processor -- start 2024-03-23T07:57:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.185+0000 7f5a18800640 1 -- start start 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.186+0000 7f5a18800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a101604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.186+0000 7f5a18800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a10070cd0 0x7f5a10160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.186+0000 7f5a18800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1006d390 con 0x7f5a101456a0 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.186+0000 7f5a18800640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a100ad890 con 0x7f5a10070cd0 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.186+0000 7f5a18800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10145a80 con 0x7f5a100cf040 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a16800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a10070cd0 0x7f5a10160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a16800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a10070cd0 0x7f5a10160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37422/0 (socket says 172.21.15.105:37422) 2024-03-23T07:57:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a16800640 1 -- 172.21.15.105:0/2301430360 learned_addr learned my addr 172.21.15.105:0/2301430360 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:30.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a15400640 1 --2- 172.21.15.105:0/2301430360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a101604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a16800640 1 -- 172.21.15.105:0/2301430360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 msgr2=0x7f5a101604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:30.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a16800640 1 --2- 172.21.15.105:0/2301430360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a101604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a16800640 1 -- 172.21.15.105:0/2301430360 >> v1:172.21.15.132:6789/0 conn(0x7f5a101456a0 legacy=0x7f5a100ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.187+0000 7f5a16800640 1 -- 172.21.15.105:0/2301430360 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1006d670 con 0x7f5a10070cd0 2024-03-23T07:57:30.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.188+0000 7f5a16800640 1 --2- 172.21.15.105:0/2301430360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a10070cd0 0x7f5a10160a00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009e80 tx=0x7f5a0c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3495bbcbd2626635 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:30.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a14a00640 1 -- 172.21.15.105:0/2301430360 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c02caf0 con 0x7f5a10070cd0 2024-03-23T07:57:30.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a14a00640 1 -- 172.21.15.105:0/2301430360 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c02cc50 con 0x7f5a10070cd0 2024-03-23T07:57:30.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a14a00640 1 -- 172.21.15.105:0/2301430360 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0343f0 con 0x7f5a10070cd0 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 -- 172.21.15.105:0/2301430360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a10070cd0 msgr2=0x7f5a10160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 --2- 172.21.15.105:0/2301430360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a10070cd0 0x7f5a10160a00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009e80 tx=0x7f5a0c02bf50 comp rx=0 tx=0).stop 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 -- 172.21.15.105:0/2301430360 shutdown_connections 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 --2- 172.21.15.105:0/2301430360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a10070cd0 0x7f5a10160a00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 --2- 172.21.15.105:0/2301430360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a101604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 -- 172.21.15.105:0/2301430360 >> 172.21.15.105:0/2301430360 conn(0x7f5a1006a0b0 msgr2=0x7f5a1006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 -- 172.21.15.105:0/2301430360 shutdown_connections 2024-03-23T07:57:30.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.189+0000 7f5a18800640 1 -- 172.21.15.105:0/2301430360 wait complete. 2024-03-23T07:57:30.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.191+0000 7f5a18800640 1 Processor -- start 2024-03-23T07:57:30.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.191+0000 7f5a18800640 1 -- start start 2024-03-23T07:57:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.192+0000 7f5a18800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a10070cd0 0x7f5a100b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.192+0000 7f5a18800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a100ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.192+0000 7f5a18800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101456a0 0x7f5a100aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.192+0000 7f5a18800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100b0f40 con 0x7f5a101456a0 2024-03-23T07:57:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.192+0000 7f5a18800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1013d400 con 0x7f5a10070cd0 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.192+0000 7f5a18800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1015cf30 con 0x7f5a100cf040 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.192+0000 7f5a16800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101456a0 0x7f5a100aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101456a0 0x7f5a100aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37428/0 (socket says 172.21.15.105:37428) 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 -- 172.21.15.105:0/785061542 learned_addr learned my addr 172.21.15.105:0/785061542 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a15400640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a100ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a15e00640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a10070cd0 0x7f5a100b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 -- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 msgr2=0x7f5a100ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a100ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 -- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a10070cd0 msgr2=0x7f5a100b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a10070cd0 0x7f5a100b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 -- 172.21.15.105:0/785061542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c009b30 con 0x7f5a101456a0 2024-03-23T07:57:30.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.193+0000 7f5a16800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101456a0 0x7f5a100aea70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c036040 tx=0x7f5a0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.194+0000 7f5a0a000640 1 -- 172.21.15.105:0/785061542 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c009510 con 0x7f5a101456a0 2024-03-23T07:57:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.194+0000 7f5a0a000640 1 -- 172.21.15.105:0/785061542 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c03e450 con 0x7f5a101456a0 2024-03-23T07:57:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.194+0000 7f5a0a000640 1 -- 172.21.15.105:0/785061542 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c03e800 con 0x7f5a101456a0 2024-03-23T07:57:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.194+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1015d210 con 0x7f5a101456a0 2024-03-23T07:57:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.194+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1015d7d0 con 0x7f5a101456a0 2024-03-23T07:57:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.195+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a10073390 con 0x7f5a101456a0 2024-03-23T07:57:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.196+0000 7f5a0a000640 1 -- 172.21.15.105:0/785061542 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5a0c03e960 con 0x7f5a101456a0 2024-03-23T07:57:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.197+0000 7f5a0a000640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59f006ac00 0x7f59f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.197+0000 7f5a0a000640 1 -- 172.21.15.105:0/785061542 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5a0c0ac470 con 0x7f5a101456a0 2024-03-23T07:57:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.197+0000 7f5a15e00640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59f006ac00 0x7f59f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.198+0000 7f5a15e00640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59f006ac00 0x7f59f006d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a04000f60 tx=0x7f5a04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.202+0000 7f5a0a000640 1 -- 172.21.15.105:0/785061542 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5a0c0772f0 con 0x7f5a101456a0 2024-03-23T07:57:30.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.396+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a100b0690 con 0x7f5a101456a0 2024-03-23T07:57:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.397+0000 7f5a0a000640 1 -- 172.21.15.105:0/785061542 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a0c07a910 con 0x7f5a101456a0 2024-03-23T07:57:30.397 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59f006ac00 msgr2=0x7f59f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59f006ac00 0x7f59f006d0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a04000f60 tx=0x7f5a04009040 comp rx=0 tx=0).stop 2024-03-23T07:57:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101456a0 msgr2=0x7f5a100aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101456a0 0x7f5a100aea70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c036040 tx=0x7f5a0c004a40 comp rx=0 tx=0).stop 2024-03-23T07:57:30.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 shutdown_connections 2024-03-23T07:57:30.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101456a0 0x7f5a100aea70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f59f006ac00 0x7f59f006d0c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a100cf040 0x7f5a100ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 --2- 172.21.15.105:0/785061542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a10070cd0 0x7f5a100b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 >> 172.21.15.105:0/785061542 conn(0x7f5a1006a0b0 msgr2=0x7f5a101422b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:30.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.399+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 shutdown_connections 2024-03-23T07:57:30.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.400+0000 7f5a18800640 1 -- 172.21.15.105:0/785061542 wait complete. 2024-03-23T07:57:30.418 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.626+0000 7f85ba400640 1 Processor -- start 2024-03-23T07:57:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.627+0000 7f85ba400640 1 -- start start 2024-03-23T07:57:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85ba400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f85b40cd740 0x7f85b40b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85ba400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85b40b5aa0 0x7f85b40b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85ba400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b40d20c0 con 0x7f85b40d0d40 2024-03-23T07:57:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85ba400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b40d14b0 con 0x7f85b40cd740 2024-03-23T07:57:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85ba400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b40d1790 con 0x7f85b40b5aa0 2024-03-23T07:57:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85b3400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f85b40cd740 0x7f85b40b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85b3400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f85b40cd740 0x7f85b40b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37438/0 (socket says 172.21.15.105:37438) 2024-03-23T07:57:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85b3400640 1 -- 172.21.15.105:0/3190360910 learned_addr learned my addr 172.21.15.105:0/3190360910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85b3400640 1 -- 172.21.15.105:0/3190360910 >> v2:172.21.15.132:3300/0 conn(0x7f85b40b5aa0 msgr2=0x7f85b40b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.628+0000 7f85b3e00640 1 --2- 172.21.15.105:0/3190360910 >> v2:172.21.15.132:3300/0 conn(0x7f85b40b5aa0 0x7f85b40b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.629+0000 7f85b3400640 1 --2- 172.21.15.105:0/3190360910 >> v2:172.21.15.132:3300/0 conn(0x7f85b40b5aa0 0x7f85b40b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.629+0000 7f85b3400640 1 -- 172.21.15.105:0/3190360910 >> v1:172.21.15.132:6789/0 conn(0x7f85b40d0d40 legacy=0x7f85b40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.629+0000 7f85b3400640 1 -- 172.21.15.105:0/3190360910 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b40d1a70 con 0x7f85b40cd740 2024-03-23T07:57:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.629+0000 7f85b3e00640 1 --2- 172.21.15.105:0/3190360910 >> v2:172.21.15.132:3300/0 conn(0x7f85b40b5aa0 0x7f85b40b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.630+0000 7f85b3400640 1 --2- 172.21.15.105:0/3190360910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40cd740 0x7f85b40b84b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f85b40c99c0 tx=0x7f85a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e91b096437b248c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.630+0000 7f85b2a00640 1 -- 172.21.15.105:0/3190360910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4031870 con 0x7f85b40cd740 2024-03-23T07:57:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.630+0000 7f85b2a00640 1 -- 172.21.15.105:0/3190360910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a40319d0 con 0x7f85b40cd740 2024-03-23T07:57:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.630+0000 7f85b2a00640 1 -- 172.21.15.105:0/3190360910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4031d60 con 0x7f85b40cd740 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 -- 172.21.15.105:0/3190360910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40cd740 msgr2=0x7f85b40b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 --2- 172.21.15.105:0/3190360910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40cd740 0x7f85b40b84b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f85b40c99c0 tx=0x7f85a402b910 comp rx=0 tx=0).stop 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 -- 172.21.15.105:0/3190360910 shutdown_connections 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 --2- 172.21.15.105:0/3190360910 >> v2:172.21.15.132:3300/0 conn(0x7f85b40b5aa0 0x7f85b40b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 --2- 172.21.15.105:0/3190360910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40cd740 0x7f85b40b84b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 -- 172.21.15.105:0/3190360910 >> 172.21.15.105:0/3190360910 conn(0x7f85b40cf5d0 msgr2=0x7f85b40cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 -- 172.21.15.105:0/3190360910 shutdown_connections 2024-03-23T07:57:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.631+0000 7f85ba400640 1 -- 172.21.15.105:0/3190360910 wait complete. 2024-03-23T07:57:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.633+0000 7f85ba400640 1 Processor -- start 2024-03-23T07:57:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.633+0000 7f85ba400640 1 -- start start 2024-03-23T07:57:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85ba400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85b40b5aa0 0x7f85b414b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85ba400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85b40cd740 0x7f85b40abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85ba400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40d0d40 0x7f85b40ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85ba400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b40a9270 con 0x7f85b40d0d40 2024-03-23T07:57:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85ba400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b414bc60 con 0x7f85b40cd740 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85ba400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b40b40b0 con 0x7f85b40b5aa0 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85b3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40d0d40 0x7f85b40ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85b3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40d0d40 0x7f85b40ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37440/0 (socket says 172.21.15.105:37440) 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85b3e00640 1 -- 172.21.15.105:0/3785876630 learned_addr learned my addr 172.21.15.105:0/3785876630 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.634+0000 7f85aaa00640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85b40cd740 0x7f85b40abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85b40b5aa0 0x7f85b414b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3e00640 1 -- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85b40b5aa0 msgr2=0x7f85b414b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3e00640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85b40b5aa0 0x7f85b414b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3e00640 1 -- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85b40cd740 msgr2=0x7f85b40abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3e00640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85b40cd740 0x7f85b40abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3e00640 1 -- 172.21.15.105:0/3785876630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a40094d0 con 0x7f85b40d0d40 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85b40b5aa0 0x7f85b414b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85b3e00640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40d0d40 0x7f85b40ac050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00cef0 tx=0x7f85ac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85abe00640 1 -- 172.21.15.105:0/3785876630 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00d840 con 0x7f85b40d0d40 2024-03-23T07:57:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85abe00640 1 -- 172.21.15.105:0/3785876630 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac00d9c0 con 0x7f85b40d0d40 2024-03-23T07:57:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85abe00640 1 -- 172.21.15.105:0/3785876630 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00dd70 con 0x7f85b40d0d40 2024-03-23T07:57:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b414c0a0 con 0x7f85b40d0d40 2024-03-23T07:57:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.635+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b414c660 con 0x7f85b40d0d40 2024-03-23T07:57:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.637+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b414bf00 con 0x7f85b40d0d40 2024-03-23T07:57:30.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.637+0000 7f85abe00640 1 -- 172.21.15.105:0/3785876630 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f85ac01e3b0 con 0x7f85b40d0d40 2024-03-23T07:57:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.638+0000 7f85abe00640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f859006ac30 0x7f859006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.638+0000 7f85abe00640 1 -- 172.21.15.105:0/3785876630 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f85ac08a080 con 0x7f85b40d0d40 2024-03-23T07:57:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.639+0000 7f85b3400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f859006ac30 0x7f859006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:30.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.640+0000 7f85b3400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f859006ac30 0x7f859006d0f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85a4036000 tx=0x7f85a403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.643+0000 7f85abe00640 1 -- 172.21.15.105:0/3785876630 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f85b414bf00 con 0x7f85b40d0d40 2024-03-23T07:57:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.836+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85b40d2530 con 0x7f85b40d0d40 2024-03-23T07:57:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.837+0000 7f85abe00640 1 -- 172.21.15.105:0/3785876630 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85b40d2530 con 0x7f85b40d0d40 2024-03-23T07:57:30.837 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f859006ac30 msgr2=0x7f859006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f859006ac30 0x7f859006d0f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85a4036000 tx=0x7f85a403c040 comp rx=0 tx=0).stop 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40d0d40 msgr2=0x7f85b40ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40d0d40 0x7f85b40ac050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00cef0 tx=0x7f85ac00a710 comp rx=0 tx=0).stop 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 shutdown_connections 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b40d0d40 0x7f85b40ac050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85b40cd740 0x7f85b40abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f859006ac30 0x7f859006d0f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 --2- 172.21.15.105:0/3785876630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85b40b5aa0 0x7f85b414b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 >> 172.21.15.105:0/3785876630 conn(0x7f85b40cf5d0 msgr2=0x7f85b415f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 shutdown_connections 2024-03-23T07:57:30.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:30.840+0000 7f85ba400640 1 -- 172.21.15.105:0/3785876630 wait complete. 2024-03-23T07:57:30.858 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:31.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.069+0000 7fcbc9000640 1 Processor -- start 2024-03-23T07:57:31.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.069+0000 7fcbc9000640 1 -- start start 2024-03-23T07:57:31.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc9000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcbc40ce930 0x7fcbc40add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc9000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cd3e0 0x7fcbc40ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc9000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc40cf250 con 0x7fcbc406ce90 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc9000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc406c760 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc9000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc40b42d0 con 0x7fcbc40ce930 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc1400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cd3e0 0x7fcbc40ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc1400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cd3e0 0x7fcbc40ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37444/0 (socket says 172.21.15.105:37444) 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.071+0000 7fcbc1400640 1 -- 172.21.15.105:0/3820819276 learned_addr learned my addr 172.21.15.105:0/3820819276 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.072+0000 7fcbc1e00640 1 --2- 172.21.15.105:0/3820819276 >> v2:172.21.15.132:3300/0 conn(0x7fcbc40ce930 0x7fcbc40add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.072+0000 7fcbc1400640 1 -- 172.21.15.105:0/3820819276 >> v2:172.21.15.132:3300/0 conn(0x7fcbc40ce930 msgr2=0x7fcbc40add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.072+0000 7fcbc1400640 1 --2- 172.21.15.105:0/3820819276 >> v2:172.21.15.132:3300/0 conn(0x7fcbc40ce930 0x7fcbc40add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.072+0000 7fcbc1400640 1 -- 172.21.15.105:0/3820819276 >> v1:172.21.15.132:6789/0 conn(0x7fcbc406ce90 legacy=0x7fcbc415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.072+0000 7fcbc1400640 1 -- 172.21.15.105:0/3820819276 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc40cdba0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.072+0000 7fcbc1e00640 1 --2- 172.21.15.105:0/3820819276 >> v2:172.21.15.132:3300/0 conn(0x7fcbc40ce930 0x7fcbc40add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:31.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.073+0000 7fcbc1400640 1 --2- 172.21.15.105:0/3820819276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc40ced10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40cf540 tx=0x7fcbac02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a2f3929bc2196e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.073+0000 7fcbc0a00640 1 -- 172.21.15.105:0/3820819276 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac031890 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.073+0000 7fcbc0a00640 1 -- 172.21.15.105:0/3820819276 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac0319f0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.073+0000 7fcbc0a00640 1 -- 172.21.15.105:0/3820819276 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac031d80 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 -- 172.21.15.105:0/3820819276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 msgr2=0x7fcbc40ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 --2- 172.21.15.105:0/3820819276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc40ced10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40cf540 tx=0x7fcbac02b930 comp rx=0 tx=0).stop 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 -- 172.21.15.105:0/3820819276 shutdown_connections 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 --2- 172.21.15.105:0/3820819276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc40ced10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 --2- 172.21.15.105:0/3820819276 >> v2:172.21.15.132:3300/0 conn(0x7fcbc40ce930 0x7fcbc40add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 -- 172.21.15.105:0/3820819276 >> 172.21.15.105:0/3820819276 conn(0x7fcbc406a0b0 msgr2=0x7fcbc406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 -- 172.21.15.105:0/3820819276 shutdown_connections 2024-03-23T07:57:31.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.074+0000 7fcbc9000640 1 -- 172.21.15.105:0/3820819276 wait complete. 2024-03-23T07:57:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.076+0000 7fcbc9000640 1 Processor -- start 2024-03-23T07:57:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.076+0000 7fcbc9000640 1 -- start start 2024-03-23T07:57:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc9000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbc406ce90 0x7fcbc4148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc9000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc4145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc9000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbc40ce930 0x7fcbc4149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc9000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc41470c0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc9000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4149110 con 0x7fcbc40ce930 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc9000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc41464b0 con 0x7fcbc406ce90 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc1400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc4145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc1400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc4145870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37456/0 (socket says 172.21.15.105:37456) 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.077+0000 7fcbc1400640 1 -- 172.21.15.105:0/1625416964 learned_addr learned my addr 172.21.15.105:0/1625416964 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc1e00640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbc406ce90 0x7fcbc4148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc1400640 1 -- 172.21.15.105:0/1625416964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbc406ce90 msgr2=0x7fcbc4148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc1400640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbc406ce90 0x7fcbc4148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc1400640 1 -- 172.21.15.105:0/1625416964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbc40ce930 msgr2=0x7fcbc4149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc1400640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbc40ce930 0x7fcbc4149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc1400640 1 -- 172.21.15.105:0/1625416964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0094d0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc1400640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc4145870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcbac000c00 tx=0x7fcbac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:31.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbba000640 1 -- 172.21.15.105:0/1625416964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac031840 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbba000640 1 -- 172.21.15.105:0/1625416964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac0443f0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbba000640 1 -- 172.21.15.105:0/1625416964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0447a0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.078+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc4146790 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.079+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc4146d50 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.080+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc4147230 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.081+0000 7fcbba000640 1 -- 172.21.15.105:0/1625416964 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcbac04d450 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.081+0000 7fcbba000640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcba006ac30 0x7fcba006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.082+0000 7fcbba000640 1 -- 172.21.15.105:0/1625416964 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcbac0abfd0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.082+0000 7fcbc1e00640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcba006ac30 0x7fcba006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.082+0000 7fcbc1e00640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcba006ac30 0x7fcba006d0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc008ee0 tx=0x7fcbbc008430 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:31.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.086+0000 7fcbba000640 1 -- 172.21.15.105:0/1625416964 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcbac076dc0 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.278+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcbc4073390 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.279+0000 7fcbba000640 1 -- 172.21.15.105:0/1625416964 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcbac035020 con 0x7fcbc40cd3e0 2024-03-23T07:57:31.279 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcba006ac30 msgr2=0x7fcba006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcba006ac30 0x7fcba006d0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc008ee0 tx=0x7fcbbc008430 comp rx=0 tx=0).stop 2024-03-23T07:57:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 msgr2=0x7fcbc4145870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc4145870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcbac000c00 tx=0x7fcbac0046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 shutdown_connections 2024-03-23T07:57:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbc40ce930 0x7fcbc4149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40cd3e0 0x7fcbc4145870 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fcba006ac30 0x7fcba006d0f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 --2- 172.21.15.105:0/1625416964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbc406ce90 0x7fcbc4148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.281+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 >> 172.21.15.105:0/1625416964 conn(0x7fcbc406a0b0 msgr2=0x7fcbc40cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.282+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 shutdown_connections 2024-03-23T07:57:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.282+0000 7fcbc9000640 1 -- 172.21.15.105:0/1625416964 wait complete. 2024-03-23T07:57:31.300 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T07:57:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.503+0000 7fa699600640 1 Processor -- start 2024-03-23T07:57:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.503+0000 7fa699600640 1 -- start start 2024-03-23T07:57:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa699600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6940ce6a0 0x7fa69406bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa699600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa694098a40 0x7fa694098e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa699600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69406c300 con 0x7fa69406b9e0 2024-03-23T07:57:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa699600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940993d0 con 0x7fa694098a40 2024-03-23T07:57:31.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa699600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940ceac0 con 0x7fa6940ce6a0 2024-03-23T07:57:31.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa693e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa69406b9e0 0x7fa69406afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60478/0 (socket says 172.21.15.105:60478) 2024-03-23T07:57:31.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa693e00640 1 -- 172.21.15.105:0/1645558709 learned_addr learned my addr 172.21.15.105:0/1645558709 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:31.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa698c00640 1 --2- 172.21.15.105:0/1645558709 >> v2:172.21.15.105:3300/0 conn(0x7fa694098a40 0x7fa694098e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa693400640 1 -- 172.21.15.105:0/1645558709 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900891416 0 0) 0x7fa69406c300 con 0x7fa69406b9e0 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa68b400640 1 --2- 172.21.15.105:0/1645558709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6940ce6a0 0x7fa69406bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.505+0000 7fa693400640 1 -- 172.21.15.105:0/1645558709 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa678003520 con 0x7fa69406b9e0 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.506+0000 7fa698c00640 1 -- 172.21.15.105:0/1645558709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6940ce6a0 msgr2=0x7fa69406bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.506+0000 7fa698c00640 1 --2- 172.21.15.105:0/1645558709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6940ce6a0 0x7fa69406bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.506+0000 7fa698c00640 1 -- 172.21.15.105:0/1645558709 >> v1:172.21.15.105:6789/0 conn(0x7fa69406b9e0 legacy=0x7fa69406afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.506+0000 7fa698c00640 1 -- 172.21.15.105:0/1645558709 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940ced60 con 0x7fa694098a40 2024-03-23T07:57:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.506+0000 7fa68b400640 1 --2- 172.21.15.105:0/1645558709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6940ce6a0 0x7fa69406bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:31.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.507+0000 7fa698c00640 1 --2- 172.21.15.105:0/1645558709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa694098a40 0x7fa694098e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0097a0 tx=0x7fa68c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe3cb8268ff5cd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:31.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.507+0000 7fa693400640 1 -- 172.21.15.105:0/1645558709 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c031780 con 0x7fa694098a40 2024-03-23T07:57:31.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.507+0000 7fa693400640 1 -- 172.21.15.105:0/1645558709 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c0318e0 con 0x7fa694098a40 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.507+0000 7fa693400640 1 -- 172.21.15.105:0/1645558709 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c031c50 con 0x7fa694098a40 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.507+0000 7fa699600640 1 -- 172.21.15.105:0/1645558709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa694098a40 msgr2=0x7fa694098e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.507+0000 7fa699600640 1 --2- 172.21.15.105:0/1645558709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa694098a40 0x7fa694098e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0097a0 tx=0x7fa68c02b820 comp rx=0 tx=0).stop 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.508+0000 7fa699600640 1 -- 172.21.15.105:0/1645558709 shutdown_connections 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.508+0000 7fa699600640 1 --2- 172.21.15.105:0/1645558709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa694098a40 0x7fa694098e90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.508+0000 7fa699600640 1 --2- 172.21.15.105:0/1645558709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6940ce6a0 0x7fa69406bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.508+0000 7fa699600640 1 -- 172.21.15.105:0/1645558709 >> 172.21.15.105:0/1645558709 conn(0x7fa6940d04d0 msgr2=0x7fa6940d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.508+0000 7fa699600640 1 -- 172.21.15.105:0/1645558709 shutdown_connections 2024-03-23T07:57:31.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.508+0000 7fa699600640 1 -- 172.21.15.105:0/1645558709 wait complete. 2024-03-23T07:57:31.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.509+0000 7fa699600640 1 Processor -- start 2024-03-23T07:57:31.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.509+0000 7fa699600640 1 -- start start 2024-03-23T07:57:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.510+0000 7fa699600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa69406b9e0 0x7fa6940b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.510+0000 7fa699600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa694098a40 0x7fa6940b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.510+0000 7fa699600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6940ce6a0 0x7fa69414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.510+0000 7fa699600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69415e260 con 0x7fa6940ce6a0 2024-03-23T07:57:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.510+0000 7fa699600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69415d650 con 0x7fa694098a40 2024-03-23T07:57:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.510+0000 7fa699600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69415d930 con 0x7fa69406b9e0 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.510+0000 7fa698c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6940ce6a0 0x7fa69414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6940ce6a0 0x7fa69414d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37480/0 (socket says 172.21.15.105:37480) 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 -- 172.21.15.105:0/2658524541 learned_addr learned my addr 172.21.15.105:0/2658524541 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 -- 172.21.15.105:0/2658524541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa69406b9e0 msgr2=0x7fa6940b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa69406b9e0 0x7fa6940b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 -- 172.21.15.105:0/2658524541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa694098a40 msgr2=0x7fa6940b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa694098a40 0x7fa6940b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 -- 172.21.15.105:0/2658524541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c009400 con 0x7fa6940ce6a0 2024-03-23T07:57:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa698c00640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6940ce6a0 0x7fa69414d950 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa68c034040 tx=0x7fa68c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa690c00640 1 -- 172.21.15.105:0/2658524541 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c032820 con 0x7fa6940ce6a0 2024-03-23T07:57:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa690c00640 1 -- 172.21.15.105:0/2658524541 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c0329a0 con 0x7fa6940ce6a0 2024-03-23T07:57:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa690c00640 1 -- 172.21.15.105:0/2658524541 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c032d50 con 0x7fa6940ce6a0 2024-03-23T07:57:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.511+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69415dc10 con 0x7fa6940ce6a0 2024-03-23T07:57:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.512+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69414b690 con 0x7fa6940ce6a0 2024-03-23T07:57:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.513+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69404e030 con 0x7fa6940ce6a0 2024-03-23T07:57:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.514+0000 7fa690c00640 1 -- 172.21.15.105:0/2658524541 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa68c04d3f0 con 0x7fa6940ce6a0 2024-03-23T07:57:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.514+0000 7fa690c00640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa67006ac00 0x7fa67006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.514+0000 7fa690c00640 1 -- 172.21.15.105:0/2658524541 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa68c0ab2b0 con 0x7fa6940ce6a0 2024-03-23T07:57:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.515+0000 7fa693e00640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa67006ac00 0x7fa67006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.515+0000 7fa693e00640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa67006ac00 0x7fa67006d0c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa684000f60 tx=0x7fa6840078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:31.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.519+0000 7fa690c00640 1 -- 172.21.15.105:0/2658524541 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa68c076150 con 0x7fa6940ce6a0 2024-03-23T07:57:31.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.727+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa69415e3a0 con 0x7fa6940ce6a0 2024-03-23T07:57:31.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.728+0000 7fa690c00640 1 -- 172.21.15.105:0/2658524541 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa68c079770 con 0x7fa6940ce6a0 2024-03-23T07:57:31.728 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-23T07:57:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa67006ac00 msgr2=0x7fa67006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa67006ac00 0x7fa67006d0c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa684000f60 tx=0x7fa6840078f0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6940ce6a0 msgr2=0x7fa69414d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6940ce6a0 0x7fa69414d950 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa68c034040 tx=0x7fa68c0046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 shutdown_connections 2024-03-23T07:57:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6940ce6a0 0x7fa69414d950 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa694098a40 0x7fa6940b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa67006ac00 0x7fa67006d0c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 --2- 172.21.15.105:0/2658524541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa69406b9e0 0x7fa6940b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.730+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 >> 172.21.15.105:0/2658524541 conn(0x7fa6940d04d0 msgr2=0x7fa694159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.731+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 shutdown_connections 2024-03-23T07:57:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:31.731+0000 7fa699600640 1 -- 172.21.15.105:0/2658524541 wait complete. 2024-03-23T07:57:31.750 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-23T07:57:31.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:31.751 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-23T07:57:31.847 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:57:31.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:31.847 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-23T07:57:31.895 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:31.896 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:31.954 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:31.954 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:32.008 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:57:32.008 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:57:32.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.275+0000 7f3eb3e00640 1 Processor -- start 2024-03-23T07:57:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.275+0000 7f3eb3e00640 1 -- start start 2024-03-23T07:57:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.276+0000 7f3eb3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3eac15a980 0x7f3eac099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.276+0000 7f3eb3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3eac0cf430 0x7f3eac098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac0cb490 con 0x7f3eac098f80 2024-03-23T07:57:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb3e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac099410 con 0x7f3eac0cf430 2024-03-23T07:57:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac15ad60 con 0x7f3eac15a980 2024-03-23T07:57:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb0a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3eac0cf430 0x7f3eac098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb0a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3eac0cf430 0x7f3eac098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37496/0 (socket says 172.21.15.105:37496) 2024-03-23T07:57:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb0a00640 1 -- 172.21.15.105:0/421914729 learned_addr learned my addr 172.21.15.105:0/421914729 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb1400640 1 --2- 172.21.15.105:0/421914729 >> v2:172.21.15.132:3300/0 conn(0x7f3eac15a980 0x7f3eac099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb0a00640 1 -- 172.21.15.105:0/421914729 >> v2:172.21.15.132:3300/0 conn(0x7f3eac15a980 msgr2=0x7f3eac099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb0a00640 1 --2- 172.21.15.105:0/421914729 >> v2:172.21.15.132:3300/0 conn(0x7f3eac15a980 0x7f3eac099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb0a00640 1 -- 172.21.15.105:0/421914729 >> v1:172.21.15.132:6789/0 conn(0x7f3eac098f80 legacy=0x7f3eac06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.277+0000 7f3eb0a00640 1 -- 172.21.15.105:0/421914729 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac06f470 con 0x7f3eac0cf430 2024-03-23T07:57:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.279+0000 7f3eb0a00640 1 --2- 172.21.15.105:0/421914729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac0cf430 0x7f3eac098a40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0996f0 tx=0x7f3e9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c856b4fec7b1eb0d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.279+0000 7f3eabe00640 1 -- 172.21.15.105:0/421914729 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98031870 con 0x7f3eac0cf430 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.279+0000 7f3eabe00640 1 -- 172.21.15.105:0/421914729 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e980319d0 con 0x7f3eac0cf430 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.279+0000 7f3eabe00640 1 -- 172.21.15.105:0/421914729 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98031d60 con 0x7f3eac0cf430 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.279+0000 7f3eb3e00640 1 -- 172.21.15.105:0/421914729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac0cf430 msgr2=0x7f3eac098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.279+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/421914729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac0cf430 0x7f3eac098a40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0996f0 tx=0x7f3e9802b910 comp rx=0 tx=0).stop 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.280+0000 7f3eb3e00640 1 -- 172.21.15.105:0/421914729 shutdown_connections 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.280+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/421914729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac0cf430 0x7f3eac098a40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.280+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/421914729 >> v2:172.21.15.132:3300/0 conn(0x7f3eac15a980 0x7f3eac099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.280+0000 7f3eb3e00640 1 -- 172.21.15.105:0/421914729 >> 172.21.15.105:0/421914729 conn(0x7f3eac0b2160 msgr2=0x7f3eac0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.280+0000 7f3eb3e00640 1 -- 172.21.15.105:0/421914729 shutdown_connections 2024-03-23T07:57:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.280+0000 7f3eb3e00640 1 -- 172.21.15.105:0/421914729 wait complete. 2024-03-23T07:57:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.281+0000 7f3eb3e00640 1 Processor -- start 2024-03-23T07:57:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.282+0000 7f3eb3e00640 1 -- start start 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.282+0000 7f3eb3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eac098f80 0x7f3eac0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.282+0000 7f3eb3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3eac0cf430 0x7f3eac0ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.282+0000 7f3eb3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac15a980 0x7f3eac145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.282+0000 7f3eb3e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac156e90 con 0x7f3eac15a980 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.282+0000 7f3eb3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0caa30 con 0x7f3eac098f80 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.282+0000 7f3eb3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0c9470 con 0x7f3eac0cf430 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac15a980 0x7f3eac145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac15a980 0x7f3eac145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37506/0 (socket says 172.21.15.105:37506) 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 -- 172.21.15.105:0/501790661 learned_addr learned my addr 172.21.15.105:0/501790661 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb0a00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3eac0cf430 0x7f3eac0ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1400640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eac098f80 0x7f3eac0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 -- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3eac0cf430 msgr2=0x7f3eac0ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3eac0cf430 0x7f3eac0ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 -- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eac098f80 msgr2=0x7f3eac0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eac098f80 0x7f3eac0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.283+0000 7f3eb1e00640 1 -- 172.21.15.105:0/501790661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980094d0 con 0x7f3eac15a980 2024-03-23T07:57:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.284+0000 7f3eb1e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac15a980 0x7f3eac145e30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c780 tx=0x7f3ea400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.284+0000 7f3ea9600640 1 -- 172.21.15.105:0/501790661 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4003d80 con 0x7f3eac15a980 2024-03-23T07:57:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.284+0000 7f3ea9600640 1 -- 172.21.15.105:0/501790661 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea400ce90 con 0x7f3eac15a980 2024-03-23T07:57:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.284+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac0c9750 con 0x7f3eac15a980 2024-03-23T07:57:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.284+0000 7f3ea9600640 1 -- 172.21.15.105:0/501790661 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40052b0 con 0x7f3eac15a980 2024-03-23T07:57:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.284+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac0c9d10 con 0x7f3eac15a980 2024-03-23T07:57:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.285+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac073390 con 0x7f3eac15a980 2024-03-23T07:57:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.286+0000 7f3ea9600640 1 -- 172.21.15.105:0/501790661 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3ea4005410 con 0x7f3eac15a980 2024-03-23T07:57:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.287+0000 7f3ea9600640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e8c06ac00 0x7f3e8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.287+0000 7f3ea9600640 1 -- 172.21.15.105:0/501790661 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3ea4088b20 con 0x7f3eac15a980 2024-03-23T07:57:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.287+0000 7f3eb1400640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e8c06ac00 0x7f3e8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.288+0000 7f3eb1400640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e8c06ac00 0x7f3e8c06d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b7c0 tx=0x7f3ea0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.292+0000 7f3ea9600640 1 -- 172.21.15.105:0/501790661 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3ea4053920 con 0x7f3eac15a980 2024-03-23T07:57:32.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.484+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3eac0cce60 con 0x7f3eac15a980 2024-03-23T07:57:32.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.486+0000 7f3ea9600640 1 -- 172.21.15.105:0/501790661 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f3ea4056f40 con 0x7f3eac15a980 2024-03-23T07:57:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:32.487 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:32.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.488+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e8c06ac00 msgr2=0x7f3e8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.488+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e8c06ac00 0x7f3e8c06d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b7c0 tx=0x7f3ea0008040 comp rx=0 tx=0).stop 2024-03-23T07:57:32.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.488+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac15a980 msgr2=0x7f3eac145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.488+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac15a980 0x7f3eac145e30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c780 tx=0x7f3ea400cc40 comp rx=0 tx=0).stop 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.488+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 shutdown_connections 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.489+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac15a980 0x7f3eac145e30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.489+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e8c06ac00 0x7f3e8c06d0c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.489+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3eac0cf430 0x7f3eac0ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.489+0000 7f3eb3e00640 1 --2- 172.21.15.105:0/501790661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eac098f80 0x7f3eac0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.489+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 >> 172.21.15.105:0/501790661 conn(0x7f3eac0b2160 msgr2=0x7f3eac0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.489+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 shutdown_connections 2024-03-23T07:57:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.489+0000 7f3eb3e00640 1 -- 172.21.15.105:0/501790661 wait complete. 2024-03-23T07:57:32.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:32.506 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:32.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.707+0000 7ffa06000640 1 Processor -- start 2024-03-23T07:57:32.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.707+0000 7ffa06000640 1 -- start start 2024-03-23T07:57:32.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.708+0000 7ffa06000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffa00098a40 0x7ffa00098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.708+0000 7ffa06000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa00160d90 0x7ffa0015c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.708+0000 7ffa06000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000d0280 con 0x7ffa0006b2e0 2024-03-23T07:57:32.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.708+0000 7ffa06000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa00099360 con 0x7ffa00160d90 2024-03-23T07:57:32.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.708+0000 7ffa06000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0015d9e0 con 0x7ffa00098a40 2024-03-23T07:57:32.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ffe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa00160d90 0x7ffa0015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ffe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa00160d90 0x7ffa0015c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37516/0 (socket says 172.21.15.105:37516) 2024-03-23T07:57:32.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ffe00640 1 -- 172.21.15.105:0/1806498983 learned_addr learned my addr 172.21.15.105:0/1806498983 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:32.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ff400640 1 --2- 172.21.15.105:0/1806498983 >> v2:172.21.15.132:3300/0 conn(0x7ffa00098a40 0x7ffa00098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ffe00640 1 -- 172.21.15.105:0/1806498983 >> v2:172.21.15.132:3300/0 conn(0x7ffa00098a40 msgr2=0x7ffa00098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:32.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ffe00640 1 --2- 172.21.15.105:0/1806498983 >> v2:172.21.15.132:3300/0 conn(0x7ffa00098a40 0x7ffa00098e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ffe00640 1 -- 172.21.15.105:0/1806498983 >> v1:172.21.15.132:6789/0 conn(0x7ffa0006b2e0 legacy=0x7ffa00160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:32.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.709+0000 7ff9ffe00640 1 -- 172.21.15.105:0/1806498983 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0015dcc0 con 0x7ffa00160d90 2024-03-23T07:57:32.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.710+0000 7ff9ffe00640 1 --2- 172.21.15.105:0/1806498983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa0015c300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa000713d0 tx=0x7ff9f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cd0d5ad91e380e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:32.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.711+0000 7ff9fe000640 1 -- 172.21.15.105:0/1806498983 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8031870 con 0x7ffa00160d90 2024-03-23T07:57:32.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.711+0000 7ff9fe000640 1 -- 172.21.15.105:0/1806498983 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f80319d0 con 0x7ffa00160d90 2024-03-23T07:57:32.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.711+0000 7ff9fe000640 1 -- 172.21.15.105:0/1806498983 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8031d60 con 0x7ffa00160d90 2024-03-23T07:57:32.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.711+0000 7ffa06000640 1 -- 172.21.15.105:0/1806498983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 msgr2=0x7ffa0015c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.711+0000 7ffa06000640 1 --2- 172.21.15.105:0/1806498983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa0015c300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa000713d0 tx=0x7ff9f802b910 comp rx=0 tx=0).stop 2024-03-23T07:57:32.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.712+0000 7ffa06000640 1 -- 172.21.15.105:0/1806498983 shutdown_connections 2024-03-23T07:57:32.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.712+0000 7ffa06000640 1 --2- 172.21.15.105:0/1806498983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa0015c300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.712+0000 7ffa06000640 1 --2- 172.21.15.105:0/1806498983 >> v2:172.21.15.132:3300/0 conn(0x7ffa00098a40 0x7ffa00098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.712+0000 7ffa06000640 1 -- 172.21.15.105:0/1806498983 >> 172.21.15.105:0/1806498983 conn(0x7ffa000b2180 msgr2=0x7ffa000b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:32.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.712+0000 7ffa06000640 1 -- 172.21.15.105:0/1806498983 shutdown_connections 2024-03-23T07:57:32.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.712+0000 7ffa06000640 1 -- 172.21.15.105:0/1806498983 wait complete. 2024-03-23T07:57:32.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.713+0000 7ffa06000640 1 Processor -- start 2024-03-23T07:57:32.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.714+0000 7ffa06000640 1 -- start start 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.714+0000 7ffa06000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa0006b2e0 0x7ffa00159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.714+0000 7ffa06000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00098a40 0x7ffa0009d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.714+0000 7ffa06000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa000b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ffa06000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001e76b0 con 0x7ffa00160d90 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ffa06000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001598d0 con 0x7ffa0006b2e0 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ffa06000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001e6a30 con 0x7ffa00098a40 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa000b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa000b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37532/0 (socket says 172.21.15.105:37532) 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 -- 172.21.15.105:0/3211065255 learned_addr learned my addr 172.21.15.105:0/3211065255 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ff400640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa0006b2e0 0x7ffa00159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 -- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00098a40 msgr2=0x7ffa0009d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00098a40 0x7ffa0009d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 -- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa0006b2e0 msgr2=0x7ffa00159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa0006b2e0 0x7ffa00159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 -- 172.21.15.105:0/3211065255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80094d0 con 0x7ffa00160d90 2024-03-23T07:57:32.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ff400640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa0006b2e0 0x7ffa00159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:32.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.715+0000 7ff9ffe00640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa000b5410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8034040 tx=0x7ff9f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.716+0000 7ff9f7400640 1 -- 172.21.15.105:0/3211065255 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f803ac20 con 0x7ffa00160d90 2024-03-23T07:57:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.716+0000 7ff9f7400640 1 -- 172.21.15.105:0/3211065255 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f803ad80 con 0x7ffa00160d90 2024-03-23T07:57:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.716+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa001e6d10 con 0x7ffa00160d90 2024-03-23T07:57:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.716+0000 7ff9f7400640 1 -- 172.21.15.105:0/3211065255 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80445f0 con 0x7ffa00160d90 2024-03-23T07:57:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.716+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001e72d0 con 0x7ffa00160d90 2024-03-23T07:57:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.717+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa001e77f0 con 0x7ffa00160d90 2024-03-23T07:57:32.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.718+0000 7ff9f7400640 1 -- 172.21.15.105:0/3211065255 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff9f804d450 con 0x7ffa00160d90 2024-03-23T07:57:32.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.719+0000 7ff9f7400640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff9dc06ac00 0x7ff9dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:32.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.719+0000 7ff9f7400640 1 -- 172.21.15.105:0/3211065255 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff9f802d040 con 0x7ffa00160d90 2024-03-23T07:57:32.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.719+0000 7ff9ff400640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff9dc06ac00 0x7ff9dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:32.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.720+0000 7ff9ff400640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff9dc06ac00 0x7ff9dc06d0c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0007330 tx=0x7ff9f000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:32.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.724+0000 7ff9f7400640 1 -- 172.21.15.105:0/3211065255 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff9f8076aa0 con 0x7ffa00160d90 2024-03-23T07:57:32.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.918+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa00066bb0 con 0x7ffa00160d90 2024-03-23T07:57:32.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.919+0000 7ff9f7400640 1 -- 172.21.15.105:0/3211065255 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9f80350a0 con 0x7ffa00160d90 2024-03-23T07:57:32.920 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.922+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff9dc06ac00 msgr2=0x7ff9dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.922+0000 7ffa06000640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff9dc06ac00 0x7ff9dc06d0c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0007330 tx=0x7ff9f000b040 comp rx=0 tx=0).stop 2024-03-23T07:57:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.922+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 msgr2=0x7ffa000b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.922+0000 7ffa06000640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa000b5410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8034040 tx=0x7ff9f80046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 shutdown_connections 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa00160d90 0x7ffa000b5410 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff9dc06ac00 0x7ff9dc06d0c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00098a40 0x7ffa0009d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 --2- 172.21.15.105:0/3211065255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa0006b2e0 0x7ffa00159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 >> 172.21.15.105:0/3211065255 conn(0x7ffa000b2180 msgr2=0x7ffa000c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 shutdown_connections 2024-03-23T07:57:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:32.923+0000 7ffa06000640 1 -- 172.21.15.105:0/3211065255 wait complete. 2024-03-23T07:57:32.941 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:33.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.154+0000 7f1cdf400640 1 Processor -- start 2024-03-23T07:57:33.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.154+0000 7f1cdf400640 1 -- start start 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.156+0000 7f1cdf400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1cd81456a0 0x7f1cd80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.156+0000 7f1cdf400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1cd8145c20 0x7f1cd80cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.156+0000 7f1cdf400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd806d390 con 0x7f1cd8070cd0 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.156+0000 7f1cdf400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd80ad890 con 0x7f1cd8145c20 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.156+0000 7f1cdf400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd8145a80 con 0x7f1cd81456a0 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdd400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1cd8145c20 0x7f1cd80cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdd400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1cd8145c20 0x7f1cd80cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37542/0 (socket says 172.21.15.105:37542) 2024-03-23T07:57:33.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdd400640 1 -- 172.21.15.105:0/2355464241 learned_addr learned my addr 172.21.15.105:0/2355464241 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:33.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdca00640 1 --2- 172.21.15.105:0/2355464241 >> v2:172.21.15.132:3300/0 conn(0x7f1cd81456a0 0x7f1cd80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdd400640 1 -- 172.21.15.105:0/2355464241 >> v2:172.21.15.132:3300/0 conn(0x7f1cd81456a0 msgr2=0x7f1cd80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:33.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdd400640 1 --2- 172.21.15.105:0/2355464241 >> v2:172.21.15.132:3300/0 conn(0x7f1cd81456a0 0x7f1cd80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdd400640 1 -- 172.21.15.105:0/2355464241 >> v1:172.21.15.132:6789/0 conn(0x7f1cd8070cd0 legacy=0x7f1cd80ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:33.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdd400640 1 -- 172.21.15.105:0/2355464241 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd806d670 con 0x7f1cd8145c20 2024-03-23T07:57:33.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.157+0000 7f1cdca00640 1 --2- 172.21.15.105:0/2355464241 >> v2:172.21.15.132:3300/0 conn(0x7f1cd81456a0 0x7f1cd80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:33.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.158+0000 7f1cdd400640 1 --2- 172.21.15.105:0/2355464241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80cee70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80cf4f0 tx=0x7f1cd002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa85cd9dc2e7c421 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:33.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cd7e00640 1 -- 172.21.15.105:0/2355464241 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd002ca80 con 0x7f1cd8145c20 2024-03-23T07:57:33.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cd7e00640 1 -- 172.21.15.105:0/2355464241 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd002cbe0 con 0x7f1cd8145c20 2024-03-23T07:57:33.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cd7e00640 1 -- 172.21.15.105:0/2355464241 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00343f0 con 0x7f1cd8145c20 2024-03-23T07:57:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 -- 172.21.15.105:0/2355464241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 msgr2=0x7f1cd80cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 --2- 172.21.15.105:0/2355464241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80cee70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80cf4f0 tx=0x7f1cd002bee0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 -- 172.21.15.105:0/2355464241 shutdown_connections 2024-03-23T07:57:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 --2- 172.21.15.105:0/2355464241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80cee70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 --2- 172.21.15.105:0/2355464241 >> v2:172.21.15.132:3300/0 conn(0x7f1cd81456a0 0x7f1cd80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 -- 172.21.15.105:0/2355464241 >> 172.21.15.105:0/2355464241 conn(0x7f1cd806a0b0 msgr2=0x7f1cd806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 -- 172.21.15.105:0/2355464241 shutdown_connections 2024-03-23T07:57:33.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.159+0000 7f1cdf400640 1 -- 172.21.15.105:0/2355464241 wait complete. 2024-03-23T07:57:33.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.161+0000 7f1cdf400640 1 Processor -- start 2024-03-23T07:57:33.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.162+0000 7f1cdf400640 1 -- start start 2024-03-23T07:57:33.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.162+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cd8070cd0 0x7f1cd815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cd81456a0 0x7f1cd815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdf400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd812eca0 con 0x7f1cd8145c20 2024-03-23T07:57:33.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdf400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd812ddd0 con 0x7f1cd81456a0 2024-03-23T07:57:33.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdf400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd812e020 con 0x7f1cd8070cd0 2024-03-23T07:57:33.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37548/0 (socket says 172.21.15.105:37548) 2024-03-23T07:57:33.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 -- 172.21.15.105:0/394344946 learned_addr learned my addr 172.21.15.105:0/394344946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 -- 172.21.15.105:0/394344946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cd8070cd0 msgr2=0x7f1cd815bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cd8070cd0 0x7f1cd815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 -- 172.21.15.105:0/394344946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cd81456a0 msgr2=0x7f1cd815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1ccfe00640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cd81456a0 0x7f1cd815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cd81456a0 0x7f1cd815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.163+0000 7f1cdd400640 1 -- 172.21.15.105:0/394344946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009b30 con 0x7f1cd8145c20 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.164+0000 7f1cdd400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80b3350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0036040 tx=0x7f1cd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.164+0000 7f1cd5600640 1 -- 172.21.15.105:0/394344946 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd003e450 con 0x7f1cd8145c20 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.164+0000 7f1cd5600640 1 -- 172.21.15.105:0/394344946 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd003e5d0 con 0x7f1cd8145c20 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.164+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd812e300 con 0x7f1cd8145c20 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.164+0000 7f1cd5600640 1 -- 172.21.15.105:0/394344946 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd003e980 con 0x7f1cd8145c20 2024-03-23T07:57:33.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.164+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd812e8c0 con 0x7f1cd8145c20 2024-03-23T07:57:33.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.165+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd8073390 con 0x7f1cd8145c20 2024-03-23T07:57:33.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.166+0000 7f1cd5600640 1 -- 172.21.15.105:0/394344946 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1cd002cd50 con 0x7f1cd8145c20 2024-03-23T07:57:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.167+0000 7f1cd5600640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cb406ac00 0x7f1cb406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.168+0000 7f1cd5600640 1 -- 172.21.15.105:0/394344946 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1cd00ac2f0 con 0x7f1cd8145c20 2024-03-23T07:57:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.168+0000 7f1cdca00640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cb406ac00 0x7f1cb406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.168+0000 7f1cdca00640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cb406ac00 0x7f1cb406d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8006fd0 tx=0x7f1cc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:33.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.172+0000 7f1cd5600640 1 -- 172.21.15.105:0/394344946 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1cd0077190 con 0x7f1cd8145c20 2024-03-23T07:57:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.352+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1cd812ede0 con 0x7f1cd8145c20 2024-03-23T07:57:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.353+0000 7f1cd5600640 1 -- 172.21.15.105:0/394344946 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1cd812ede0 con 0x7f1cd8145c20 2024-03-23T07:57:33.354 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cb406ac00 msgr2=0x7f1cb406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cb406ac00 0x7f1cb406d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8006fd0 tx=0x7f1cc8008040 comp rx=0 tx=0).stop 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 msgr2=0x7f1cd80b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80b3350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0036040 tx=0x7f1cd0004a40 comp rx=0 tx=0).stop 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 shutdown_connections 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cd8145c20 0x7f1cd80b3350 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cd81456a0 0x7f1cd815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f1cb406ac00 0x7f1cb406d0c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 --2- 172.21.15.105:0/394344946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cd8070cd0 0x7f1cd815bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 >> 172.21.15.105:0/394344946 conn(0x7f1cd806a0b0 msgr2=0x7f1cd80ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 shutdown_connections 2024-03-23T07:57:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.356+0000 7f1cdf400640 1 -- 172.21.15.105:0/394344946 wait complete. 2024-03-23T07:57:33.373 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.578+0000 7f4649000640 1 Processor -- start 2024-03-23T07:57:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.578+0000 7f4649000640 1 -- start start 2024-03-23T07:57:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.580+0000 7f4649000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46440ce930 0x7f46440add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.580+0000 7f4649000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f46440cd3e0 0x7f46440ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.580+0000 7f4649000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f464406ce90 0x7f464406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.580+0000 7f4649000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464406c760 con 0x7f46440ce930 2024-03-23T07:57:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.580+0000 7f4649000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46440b42d0 con 0x7f46440cd3e0 2024-03-23T07:57:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.580+0000 7f4649000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46440cf250 con 0x7f464406ce90 2024-03-23T07:57:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46440ce930 0x7f46440add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46440ce930 0x7f46440add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37552/0 (socket says 172.21.15.105:37552) 2024-03-23T07:57:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 -- 172.21.15.105:0/1110502590 learned_addr learned my addr 172.21.15.105:0/1110502590 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 -- 172.21.15.105:0/1110502590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f464406ce90 msgr2=0x7f464406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 --2- 172.21.15.105:0/1110502590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f464406ce90 0x7f464406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 -- 172.21.15.105:0/1110502590 >> v2:172.21.15.132:3300/0 conn(0x7f46440cd3e0 msgr2=0x7f46440ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641400640 1 --2- 172.21.15.105:0/1110502590 >> v2:172.21.15.132:3300/0 conn(0x7f46440cd3e0 0x7f46440ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 --2- 172.21.15.105:0/1110502590 >> v2:172.21.15.132:3300/0 conn(0x7f46440cd3e0 0x7f46440ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.581+0000 7f4641e00640 1 -- 172.21.15.105:0/1110502590 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46440cdba0 con 0x7f46440ce930 2024-03-23T07:57:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.582+0000 7f4641e00640 1 --2- 172.21.15.105:0/1110502590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440ce930 0x7f46440add30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f46440cf4f0 tx=0x7f463402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829db528754c2715 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4640a00640 1 -- 172.21.15.105:0/1110502590 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634031870 con 0x7f46440ce930 2024-03-23T07:57:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4640a00640 1 -- 172.21.15.105:0/1110502590 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46340319d0 con 0x7f46440ce930 2024-03-23T07:57:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4640a00640 1 -- 172.21.15.105:0/1110502590 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634031d60 con 0x7f46440ce930 2024-03-23T07:57:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4649000640 1 -- 172.21.15.105:0/1110502590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440ce930 msgr2=0x7f46440add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4649000640 1 --2- 172.21.15.105:0/1110502590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440ce930 0x7f46440add30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f46440cf4f0 tx=0x7f463402b910 comp rx=0 tx=0).stop 2024-03-23T07:57:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4649000640 1 -- 172.21.15.105:0/1110502590 shutdown_connections 2024-03-23T07:57:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4649000640 1 --2- 172.21.15.105:0/1110502590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f464406ce90 0x7f464406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4649000640 1 --2- 172.21.15.105:0/1110502590 >> v2:172.21.15.132:3300/0 conn(0x7f46440cd3e0 0x7f46440ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4649000640 1 --2- 172.21.15.105:0/1110502590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440ce930 0x7f46440add30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.583+0000 7f4649000640 1 -- 172.21.15.105:0/1110502590 >> 172.21.15.105:0/1110502590 conn(0x7f464406a0b0 msgr2=0x7f464406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.584+0000 7f4649000640 1 -- 172.21.15.105:0/1110502590 shutdown_connections 2024-03-23T07:57:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.584+0000 7f4649000640 1 -- 172.21.15.105:0/1110502590 wait complete. 2024-03-23T07:57:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.586+0000 7f4649000640 1 Processor -- start 2024-03-23T07:57:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.586+0000 7f4649000640 1 -- start start 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4649000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464406ce90 0x7f464414f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4649000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46440cd3e0 0x7f4644148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4649000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46440ce930 0x7f464414d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4649000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46441507a0 con 0x7f464406ce90 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4649000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46441494a0 con 0x7f46440cd3e0 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4649000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464414fb90 con 0x7f46440ce930 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4641e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464406ce90 0x7f464414f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4641e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464406ce90 0x7f464414f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37568/0 (socket says 172.21.15.105:37568) 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.587+0000 7f4641e00640 1 -- 172.21.15.105:0/1202083102 learned_addr learned my addr 172.21.15.105:0/1202083102 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4641400640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46440cd3e0 0x7f4644148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4642800640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46440ce930 0x7f464414d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4641e00640 1 -- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46440ce930 msgr2=0x7f464414d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4641e00640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46440ce930 0x7f464414d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4641e00640 1 -- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46440cd3e0 msgr2=0x7f4644148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4641e00640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46440cd3e0 0x7f4644148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4641e00640 1 -- 172.21.15.105:0/1202083102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340094d0 con 0x7f464406ce90 2024-03-23T07:57:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.588+0000 7f4641e00640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464406ce90 0x7f464414f520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4634000c00 tx=0x7f46340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.589+0000 7f463a000640 1 -- 172.21.15.105:0/1202083102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634031840 con 0x7f464406ce90 2024-03-23T07:57:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.589+0000 7f463a000640 1 -- 172.21.15.105:0/1202083102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46340443f0 con 0x7f464406ce90 2024-03-23T07:57:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.589+0000 7f463a000640 1 -- 172.21.15.105:0/1202083102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46340447a0 con 0x7f464406ce90 2024-03-23T07:57:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.589+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464414fe70 con 0x7f464406ce90 2024-03-23T07:57:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.589+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644150430 con 0x7f464406ce90 2024-03-23T07:57:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.590+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4644073390 con 0x7f464406ce90 2024-03-23T07:57:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.591+0000 7f463a000640 1 -- 172.21.15.105:0/1202083102 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f463404d450 con 0x7f464406ce90 2024-03-23T07:57:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.592+0000 7f463a000640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f462406ac00 0x7f462406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.592+0000 7f463a000640 1 -- 172.21.15.105:0/1202083102 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f463402d040 con 0x7f464406ce90 2024-03-23T07:57:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.592+0000 7f4641400640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f462406ac00 0x7f462406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.593+0000 7f4641400640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f462406ac00 0x7f462406d0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4630006fd0 tx=0x7f4630008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.597+0000 7f463a000640 1 -- 172.21.15.105:0/1202083102 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4634076d20 con 0x7f464406ce90 2024-03-23T07:57:33.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.792+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f464406c540 con 0x7f464406ce90 2024-03-23T07:57:33.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.793+0000 7f463a000640 1 -- 172.21.15.105:0/1202083102 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4634035020 con 0x7f464406ce90 2024-03-23T07:57:33.794 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f462406ac00 msgr2=0x7f462406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f462406ac00 0x7f462406d0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4630006fd0 tx=0x7f4630008040 comp rx=0 tx=0).stop 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464406ce90 msgr2=0x7f464414f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464406ce90 0x7f464414f520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4634000c00 tx=0x7f46340046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 shutdown_connections 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f462406ac00 0x7f462406d0c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46440ce930 0x7f464414d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46440cd3e0 0x7f4644148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 --2- 172.21.15.105:0/1202083102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464406ce90 0x7f464414f520 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 >> 172.21.15.105:0/1202083102 conn(0x7f464406a0b0 msgr2=0x7f4644153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 shutdown_connections 2024-03-23T07:57:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:33.796+0000 7f4649000640 1 -- 172.21.15.105:0/1202083102 wait complete. 2024-03-23T07:57:33.817 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:34.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.031+0000 7f7513000640 1 Processor -- start 2024-03-23T07:57:34.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.032+0000 7f7513000640 1 -- start start 2024-03-23T07:57:34.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7513000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f750c0cd690 0x7f750c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7513000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f750c0b58f0 0x7f750c0b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7513000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750c0d2260 con 0x7f750c0ceff0 2024-03-23T07:57:34.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7513000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750c0b84a0 con 0x7f750c0cd690 2024-03-23T07:57:34.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7513000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750c0d1650 con 0x7f750c0b58f0 2024-03-23T07:57:34.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7511200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f750c0cd690 0x7f750c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7511200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f750c0cd690 0x7f750c0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37580/0 (socket says 172.21.15.105:37580) 2024-03-23T07:57:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7511200640 1 -- 172.21.15.105:0/116174583 learned_addr learned my addr 172.21.15.105:0/116174583 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.033+0000 7f7512600640 1 --2- 172.21.15.105:0/116174583 >> v2:172.21.15.132:3300/0 conn(0x7f750c0b58f0 0x7f750c0b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.034+0000 7f7511200640 1 -- 172.21.15.105:0/116174583 >> v2:172.21.15.132:3300/0 conn(0x7f750c0b58f0 msgr2=0x7f750c0b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.034+0000 7f7511200640 1 --2- 172.21.15.105:0/116174583 >> v2:172.21.15.132:3300/0 conn(0x7f750c0b58f0 0x7f750c0b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.034+0000 7f7511200640 1 -- 172.21.15.105:0/116174583 >> v1:172.21.15.132:6789/0 conn(0x7f750c0ceff0 legacy=0x7f750c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.034+0000 7f7511200640 1 -- 172.21.15.105:0/116174583 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c0d1930 con 0x7f750c0cd690 2024-03-23T07:57:34.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.035+0000 7f7511200640 1 --2- 172.21.15.105:0/116174583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0cd690 0x7f750c0cdaf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f750c0d0260 tx=0x7f74f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324fac96d059df8e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:34.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.035+0000 7f750be00640 1 -- 172.21.15.105:0/116174583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8031870 con 0x7f750c0cd690 2024-03-23T07:57:34.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.035+0000 7f750be00640 1 -- 172.21.15.105:0/116174583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f80319d0 con 0x7f750c0cd690 2024-03-23T07:57:34.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.035+0000 7f750be00640 1 -- 172.21.15.105:0/116174583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8031d60 con 0x7f750c0cd690 2024-03-23T07:57:34.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 -- 172.21.15.105:0/116174583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0cd690 msgr2=0x7f750c0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 --2- 172.21.15.105:0/116174583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0cd690 0x7f750c0cdaf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f750c0d0260 tx=0x7f74f802b910 comp rx=0 tx=0).stop 2024-03-23T07:57:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 -- 172.21.15.105:0/116174583 shutdown_connections 2024-03-23T07:57:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 --2- 172.21.15.105:0/116174583 >> v2:172.21.15.132:3300/0 conn(0x7f750c0b58f0 0x7f750c0b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 --2- 172.21.15.105:0/116174583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0cd690 0x7f750c0cdaf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 -- 172.21.15.105:0/116174583 >> 172.21.15.105:0/116174583 conn(0x7f750c0713c0 msgr2=0x7f750c0717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 -- 172.21.15.105:0/116174583 shutdown_connections 2024-03-23T07:57:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.036+0000 7f7513000640 1 -- 172.21.15.105:0/116174583 wait complete. 2024-03-23T07:57:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.038+0000 7f7513000640 1 Processor -- start 2024-03-23T07:57:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.038+0000 7f7513000640 1 -- start start 2024-03-23T07:57:34.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7513000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f750c0b58f0 0x7f750c159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7513000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f750c0cd690 0x7f750c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7513000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0ceff0 0x7f750c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7513000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c0b2d60 con 0x7f750c0ceff0 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7513000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c0b71e0 con 0x7f750c0b58f0 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7513000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750c0b23c0 con 0x7f750c0cd690 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0ceff0 0x7f750c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0ceff0 0x7f750c0c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37594/0 (socket says 172.21.15.105:37594) 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 -- 172.21.15.105:0/2997284691 learned_addr learned my addr 172.21.15.105:0/2997284691 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 -- 172.21.15.105:0/2997284691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f750c0cd690 msgr2=0x7f750c0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f750c0cd690 0x7f750c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 -- 172.21.15.105:0/2997284691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f750c0b58f0 msgr2=0x7f750c159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f750c0b58f0 0x7f750c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.039+0000 7f7512600640 1 -- 172.21.15.105:0/2997284691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f80094d0 con 0x7f750c0ceff0 2024-03-23T07:57:34.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.040+0000 7f7512600640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0ceff0 0x7f750c0c6830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f750400d5d0 tx=0x7f750400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:34.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.040+0000 7f7509600640 1 -- 172.21.15.105:0/2997284691 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040125f0 con 0x7f750c0ceff0 2024-03-23T07:57:34.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.040+0000 7f7509600640 1 -- 172.21.15.105:0/2997284691 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504012770 con 0x7f750c0ceff0 2024-03-23T07:57:34.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.040+0000 7f7509600640 1 -- 172.21.15.105:0/2997284691 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504012b20 con 0x7f750c0ceff0 2024-03-23T07:57:34.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.040+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c0b26a0 con 0x7f750c0ceff0 2024-03-23T07:57:34.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.040+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c159dc0 con 0x7f750c0ceff0 2024-03-23T07:57:34.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.042+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750c0ce090 con 0x7f750c0ceff0 2024-03-23T07:57:34.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.042+0000 7f7509600640 1 -- 172.21.15.105:0/2997284691 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7504012c80 con 0x7f750c0ceff0 2024-03-23T07:57:34.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.043+0000 7f7509600640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f74ec06ac00 0x7f74ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.043+0000 7f7509600640 1 -- 172.21.15.105:0/2997284691 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7504089c20 con 0x7f750c0ceff0 2024-03-23T07:57:34.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.043+0000 7f7511c00640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f74ec06ac00 0x7f74ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.044+0000 7f7511c00640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f74ec06ac00 0x7f74ec06d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0043c0 tx=0x7f74fc007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:34.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.048+0000 7f7509600640 1 -- 172.21.15.105:0/2997284691 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7504054a20 con 0x7f750c0ceff0 2024-03-23T07:57:34.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.242+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f750c04e0a0 con 0x7f750c0ceff0 2024-03-23T07:57:34.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.242+0000 7f7509600640 1 -- 172.21.15.105:0/2997284691 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7504058040 con 0x7f750c0ceff0 2024-03-23T07:57:34.243 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.245+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f74ec06ac00 msgr2=0x7f74ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.245+0000 7f7513000640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f74ec06ac00 0x7f74ec06d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0043c0 tx=0x7f74fc007d30 comp rx=0 tx=0).stop 2024-03-23T07:57:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.245+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0ceff0 msgr2=0x7f750c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.245+0000 7f7513000640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0ceff0 0x7f750c0c6830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f750400d5d0 tx=0x7f750400da90 comp rx=0 tx=0).stop 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 shutdown_connections 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f750c0ceff0 0x7f750c0c6830 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f74ec06ac00 0x7f74ec06d0c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f750c0cd690 0x7f750c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 --2- 172.21.15.105:0/2997284691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f750c0b58f0 0x7f750c159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 >> 172.21.15.105:0/2997284691 conn(0x7f750c0713c0 msgr2=0x7f750c0b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 shutdown_connections 2024-03-23T07:57:34.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.246+0000 7f7513000640 1 -- 172.21.15.105:0/2997284691 wait complete. 2024-03-23T07:57:34.263 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T07:57:34.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.471+0000 7f4b2b000640 1 Processor -- start 2024-03-23T07:57:34.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.471+0000 7f4b2b000640 1 -- start start 2024-03-23T07:57:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b2b000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b240cb660 0x7f4b240cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b2b000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b2406c2b0 0x7f4b2406c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b2b000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2415ae60 con 0x7f4b240ce930 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b2b000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2415bae0 con 0x7f4b2406c2b0 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b2b000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2415b140 con 0x7f4b240cb660 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b29000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b2406c2b0 0x7f4b2406c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b29000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b2406c2b0 0x7f4b2406c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37598/0 (socket says 172.21.15.105:37598) 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.473+0000 7f4b29000640 1 -- 172.21.15.105:0/1597941535 learned_addr learned my addr 172.21.15.105:0/1597941535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.474+0000 7f4b23400640 1 --2- 172.21.15.105:0/1597941535 >> v2:172.21.15.132:3300/0 conn(0x7f4b240cb660 0x7f4b240cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.474+0000 7f4b29000640 1 -- 172.21.15.105:0/1597941535 >> v2:172.21.15.132:3300/0 conn(0x7f4b240cb660 msgr2=0x7f4b240cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.474+0000 7f4b29000640 1 --2- 172.21.15.105:0/1597941535 >> v2:172.21.15.132:3300/0 conn(0x7f4b240cb660 0x7f4b240cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.474+0000 7f4b29000640 1 -- 172.21.15.105:0/1597941535 >> v1:172.21.15.132:6789/0 conn(0x7f4b240ce930 legacy=0x7f4b240ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.474+0000 7f4b29000640 1 -- 172.21.15.105:0/1597941535 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2415b420 con 0x7f4b2406c2b0 2024-03-23T07:57:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.475+0000 7f4b29000640 1 --2- 172.21.15.105:0/1597941535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2406c720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406fe60 tx=0x7f4b1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2941ba1e90d79f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.475+0000 7f4b22a00640 1 -- 172.21.15.105:0/1597941535 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c031870 con 0x7f4b2406c2b0 2024-03-23T07:57:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.475+0000 7f4b22a00640 1 -- 172.21.15.105:0/1597941535 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c0319d0 con 0x7f4b2406c2b0 2024-03-23T07:57:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.475+0000 7f4b22a00640 1 -- 172.21.15.105:0/1597941535 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c031d60 con 0x7f4b2406c2b0 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 -- 172.21.15.105:0/1597941535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 msgr2=0x7f4b2406c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 --2- 172.21.15.105:0/1597941535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2406c720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406fe60 tx=0x7f4b1c02b910 comp rx=0 tx=0).stop 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 -- 172.21.15.105:0/1597941535 shutdown_connections 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 --2- 172.21.15.105:0/1597941535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2406c720 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 --2- 172.21.15.105:0/1597941535 >> v2:172.21.15.132:3300/0 conn(0x7f4b240cb660 0x7f4b240cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 -- 172.21.15.105:0/1597941535 >> 172.21.15.105:0/1597941535 conn(0x7f4b2406a0b0 msgr2=0x7f4b2406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 -- 172.21.15.105:0/1597941535 shutdown_connections 2024-03-23T07:57:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.476+0000 7f4b2b000640 1 -- 172.21.15.105:0/1597941535 wait complete. 2024-03-23T07:57:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.478+0000 7f4b2b000640 1 Processor -- start 2024-03-23T07:57:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.478+0000 7f4b2b000640 1 -- start start 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b2b000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b2b000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b240cb660 0x7f4b241f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b2b000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240ce930 0x7f4b2415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b2b000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2414eca0 con 0x7f4b2406c2b0 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b2b000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2413be90 con 0x7f4b240cb660 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b2b000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2414f520 con 0x7f4b240ce930 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2414f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2414f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37614/0 (socket says 172.21.15.105:37614) 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.479+0000 7f4b23e00640 1 -- 172.21.15.105:0/291381315 learned_addr learned my addr 172.21.15.105:0/291381315 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b23400640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b240cb660 0x7f4b241f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b29000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240ce930 0x7f4b2415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b23e00640 1 -- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240ce930 msgr2=0x7f4b2415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b23e00640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240ce930 0x7f4b2415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b23e00640 1 -- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b240cb660 msgr2=0x7f4b241f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b23e00640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b240cb660 0x7f4b241f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b23e00640 1 -- 172.21.15.105:0/291381315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0094d0 con 0x7f4b2406c2b0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b29000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240ce930 0x7f4b2415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b23e00640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2414f9b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b3e0 tx=0x7f4b0c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b1be00640 1 -- 172.21.15.105:0/291381315 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011830 con 0x7f4b2406c2b0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b1be00640 1 -- 172.21.15.105:0/291381315 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c0119b0 con 0x7f4b2406c2b0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b1be00640 1 -- 172.21.15.105:0/291381315 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011d60 con 0x7f4b2406c2b0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.480+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b24155280 con 0x7f4b2406c2b0 2024-03-23T07:57:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.481+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2414eed0 con 0x7f4b2406c2b0 2024-03-23T07:57:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.482+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b24073390 con 0x7f4b2406c2b0 2024-03-23T07:57:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.483+0000 7f4b1be00640 1 -- 172.21.15.105:0/291381315 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4b0c01a400 con 0x7f4b2406c2b0 2024-03-23T07:57:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.483+0000 7f4b1be00640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b0006ac00 0x7f4b0006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.483+0000 7f4b1be00640 1 -- 172.21.15.105:0/291381315 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4b0c089a00 con 0x7f4b2406c2b0 2024-03-23T07:57:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.484+0000 7f4b23400640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b0006ac00 0x7f4b0006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.484+0000 7f4b23400640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b0006ac00 0x7f4b0006d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b241559a0 tx=0x7f4b140079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.488+0000 7f4b1be00640 1 -- 172.21.15.105:0/291381315 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4b0c054760 con 0x7f4b2406c2b0 2024-03-23T07:57:34.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.689+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4b2415def0 con 0x7f4b2406c2b0 2024-03-23T07:57:34.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.690+0000 7f4b1be00640 1 -- 172.21.15.105:0/291381315 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4b0c015020 con 0x7f4b2406c2b0 2024-03-23T07:57:34.691 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b0006ac00 msgr2=0x7f4b0006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b0006ac00 0x7f4b0006d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b241559a0 tx=0x7f4b140079a0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 msgr2=0x7f4b2414f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2414f9b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b3e0 tx=0x7f4b0c00b8a0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 shutdown_connections 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b0006ac00 0x7f4b0006d0c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240ce930 0x7f4b2415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b240cb660 0x7f4b241f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 --2- 172.21.15.105:0/291381315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2406c2b0 0x7f4b2414f9b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 >> 172.21.15.105:0/291381315 conn(0x7f4b2406a0b0 msgr2=0x7f4b241597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:34.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 shutdown_connections 2024-03-23T07:57:34.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:34.693+0000 7f4b2b000640 1 -- 172.21.15.105:0/291381315 wait complete. 2024-03-23T07:57:34.710 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-23T07:57:34.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:34.710 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-23T07:57:34.817 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:57:34.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:57:34.818 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-23T07:57:34.864 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-23T07:57:34.864 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:57:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.075+0000 7f4b8f000640 1 Processor -- start 2024-03-23T07:57:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.075+0000 7f4b8f000640 1 -- start start 2024-03-23T07:57:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b8f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 0x7f4b8815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b8f000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b8815d410 0x7f4b880b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b8f000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8806c6f0 con 0x7f4b880b39b0 2024-03-23T07:57:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b8f000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880b4370 con 0x7f4b8815d410 2024-03-23T07:57:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b8f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8806c9d0 con 0x7f4b8815b560 2024-03-23T07:57:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b87400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4b880b39b0 0x7f4b8815e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60494/0 (socket says 172.21.15.105:60494) 2024-03-23T07:57:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b87400640 1 -- 172.21.15.105:0/3601491618 learned_addr learned my addr 172.21.15.105:0/3601491618 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.077+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895178966 0 0) 0x7f4b8806c6f0 con 0x7f4b880b39b0 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b8d000640 1 --2- 172.21.15.105:0/3601491618 >> v2:172.21.15.132:3300/0 conn(0x7f4b8815d410 0x7f4b880b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b87e00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 0x7f4b8815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b6c003520 con 0x7f4b880b39b0 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935457615 0 0) 0x7f4b6c003520 con 0x7f4b880b39b0 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 msgr2=0x7f4b8815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 0x7f4b8815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 >> v2:172.21.15.132:3300/0 conn(0x7f4b8815d410 msgr2=0x7f4b880b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 --2- 172.21.15.105:0/3601491618 >> v2:172.21.15.132:3300/0 conn(0x7f4b8815d410 0x7f4b880b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8806ccb0 con 0x7f4b880b39b0 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f4b74001670 con 0x7f4b880b39b0 2024-03-23T07:57:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 >> v1:172.21.15.105:6789/0 conn(0x7f4b880b39b0 legacy=0x7f4b8815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b6c003970 0x7f4b6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 0x7f4b6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c004d00 0x7f4b6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880b4370 con 0x7f4b6c003970 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8806c9d0 con 0x7f4b6c004d00 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c00ba30 con 0x7f4b8815b560 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.078+0000 7f4b87e00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b6c003970 0x7f4b6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.079+0000 7f4b87400640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 0x7f4b6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.079+0000 7f4b87e00640 1 -- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 msgr2=0x7f4b6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.079+0000 7f4b8d000640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c004d00 0x7f4b6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.079+0000 7f4b87e00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8815b560 0x7f4b6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.079+0000 7f4b87e00640 1 -- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c004d00 msgr2=0x7f4b6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.079+0000 7f4b87e00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c004d00 0x7f4b6c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.079+0000 7f4b87e00640 1 -- 172.21.15.105:0/3601491618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c003520 con 0x7f4b6c003970 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.080+0000 7f4b87e00640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b6c003970 0x7f4b6c003d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ba60 tx=0x7f4b6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf0265a170bd462 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.080+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78032630 con 0x7f4b6c003970 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.080+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78032790 con 0x7f4b6c003970 2024-03-23T07:57:35.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.080+0000 7f4b86a00640 1 -- 172.21.15.105:0/3601491618 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78032b20 con 0x7f4b6c003970 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.081+0000 7f4b8f000640 1 -- 172.21.15.105:0/3601491618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b6c003970 msgr2=0x7f4b6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.081+0000 7f4b8f000640 1 --2- 172.21.15.105:0/3601491618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b6c003970 0x7f4b6c003d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ba60 tx=0x7f4b6c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.081+0000 7f4b8d000640 1 -- 172.21.15.105:0/3601491618 reap_dead start 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.081+0000 7f4b8f000640 1 -- 172.21.15.105:0/3601491618 shutdown_connections 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.081+0000 7f4b8f000640 1 -- 172.21.15.105:0/3601491618 >> 172.21.15.105:0/3601491618 conn(0x7f4b8806a3d0 msgr2=0x7f4b880b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.081+0000 7f4b8f000640 1 -- 172.21.15.105:0/3601491618 shutdown_connections 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.081+0000 7f4b8f000640 1 -- 172.21.15.105:0/3601491618 wait complete. 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.083+0000 7f4b8f000640 1 Processor -- start 2024-03-23T07:57:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.083+0000 7f4b8f000640 1 -- start start 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b8f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c003970 0x7f4b88139b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b8f000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b8813a590 0x7f4b88135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b8f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b880afde0 0x7f4b880b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b8f000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8813a0c0 con 0x7f4b8813a590 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b8f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880b07b0 con 0x7f4b6c003970 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b8f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8813d2a0 con 0x7f4b880afde0 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b87400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b8813a590 0x7f4b88135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b87400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b8813a590 0x7f4b88135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37630/0 (socket says 172.21.15.105:37630) 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b87400640 1 -- 172.21.15.105:0/145001740 learned_addr learned my addr 172.21.15.105:0/145001740 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.084+0000 7f4b87e00640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c003970 0x7f4b88139b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b87400640 1 -- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b880afde0 msgr2=0x7f4b880b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b87400640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b880afde0 0x7f4b880b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b87400640 1 -- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c003970 msgr2=0x7f4b88139b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b87400640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c003970 0x7f4b88139b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b87400640 1 -- 172.21.15.105:0/145001740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b881395e0 con 0x7f4b8813a590 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b87e00640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c003970 0x7f4b88139b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b87400640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b8813a590 0x7f4b88135560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b74001cc0 tx=0x7f4b7400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b7fe00640 1 -- 172.21.15.105:0/145001740 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400c870 con 0x7f4b8813a590 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b7fe00640 1 -- 172.21.15.105:0/145001740 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7400c9d0 con 0x7f4b8813a590 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b7fe00640 1 -- 172.21.15.105:0/145001740 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400cd80 con 0x7f4b8813a590 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8813b0a0 con 0x7f4b8813a590 2024-03-23T07:57:35.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.085+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8813adc0 con 0x7f4b8813a590 2024-03-23T07:57:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.086+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8813d100 con 0x7f4b8813a590 2024-03-23T07:57:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.087+0000 7f4b7fe00640 1 -- 172.21.15.105:0/145001740 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4b74013420 con 0x7f4b8813a590 2024-03-23T07:57:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.088+0000 7f4b7fe00640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b6806ac80 0x7f4b6806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.088+0000 7f4b7fe00640 1 -- 172.21.15.105:0/145001740 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4b740893c0 con 0x7f4b8813a590 2024-03-23T07:57:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.088+0000 7f4b87e00640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b6806ac80 0x7f4b6806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.089+0000 7f4b87e00640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b6806ac80 0x7f4b6806d140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ba30 tx=0x7f4b78030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.092+0000 7f4b7fe00640 1 -- 172.21.15.105:0/145001740 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4b8813d100 con 0x7f4b8813a590 2024-03-23T07:57:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.330+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b8813d100 con 0x7f4b8813a590 2024-03-23T07:57:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.331+0000 7f4b7fe00640 1 -- 172.21.15.105:0/145001740 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7f4b8813d100 con 0x7f4b8813a590 2024-03-23T07:57:35.332 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:35.332 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:57:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b6806ac80 msgr2=0x7f4b6806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b6806ac80 0x7f4b6806d140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ba30 tx=0x7f4b78030000 comp rx=0 tx=0).stop 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b8813a590 msgr2=0x7f4b88135560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b8813a590 0x7f4b88135560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b74001cc0 tx=0x7f4b7400ed50 comp rx=0 tx=0).stop 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 shutdown_connections 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4b6806ac80 0x7f4b6806d140 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b880afde0 0x7f4b880b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b8813a590 0x7f4b88135560 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 --2- 172.21.15.105:0/145001740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c003970 0x7f4b88139b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 >> 172.21.15.105:0/145001740 conn(0x7f4b8806a3d0 msgr2=0x7f4b880cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 shutdown_connections 2024-03-23T07:57:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.334+0000 7f4b8f000640 1 -- 172.21.15.105:0/145001740 wait complete. 2024-03-23T07:57:35.354 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.558+0000 7f4ebde00640 1 Processor -- start 2024-03-23T07:57:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.559+0000 7f4ebde00640 1 -- start start 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb814aa40 0x7f4eb80cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4ebde00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4eb80cc7b0 0x7f4eb80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4ebde00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eb80b27e0 0x7f4eb80b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4ebde00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb806da70 con 0x7f4eb80b27e0 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4ebde00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb80b1d00 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4ebde00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80b1fe0 con 0x7f4eb814aa40 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4eb7400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eb80b27e0 0x7f4eb80b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.560+0000 7f4eb7400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eb80b27e0 0x7f4eb80b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37632/0 (socket says 172.21.15.105:37632) 2024-03-23T07:57:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb7400640 1 -- 172.21.15.105:0/1402884915 learned_addr learned my addr 172.21.15.105:0/1402884915 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb6a00640 1 --2- 172.21.15.105:0/1402884915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb814aa40 0x7f4eb80cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eae000640 1 --2- 172.21.15.105:0/1402884915 >> v2:172.21.15.132:3300/0 conn(0x7f4eb80cc7b0 0x7f4eb80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb7400640 1 -- 172.21.15.105:0/1402884915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb814aa40 msgr2=0x7f4eb80cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb7400640 1 --2- 172.21.15.105:0/1402884915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb814aa40 0x7f4eb80cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb7400640 1 -- 172.21.15.105:0/1402884915 >> v2:172.21.15.132:3300/0 conn(0x7f4eb80cc7b0 msgr2=0x7f4eb80ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb7400640 1 --2- 172.21.15.105:0/1402884915 >> v2:172.21.15.132:3300/0 conn(0x7f4eb80cc7b0 0x7f4eb80ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb7400640 1 -- 172.21.15.105:0/1402884915 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80b22c0 con 0x7f4eb80b27e0 2024-03-23T07:57:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eb6a00640 1 --2- 172.21.15.105:0/1402884915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb814aa40 0x7f4eb80cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.561+0000 7f4eae000640 1 --2- 172.21.15.105:0/1402884915 >> v2:172.21.15.132:3300/0 conn(0x7f4eb80cc7b0 0x7f4eb80ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.562+0000 7f4eb7400640 1 --2- 172.21.15.105:0/1402884915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80b27e0 0x7f4eb80b2c50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8071470 tx=0x7f4eb002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a018fef511b860f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.562+0000 7f4eb6000640 1 -- 172.21.15.105:0/1402884915 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb002ca80 con 0x7f4eb80b27e0 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.562+0000 7f4eb6000640 1 -- 172.21.15.105:0/1402884915 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb002cbe0 con 0x7f4eb80b27e0 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.562+0000 7f4eb6000640 1 -- 172.21.15.105:0/1402884915 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00343f0 con 0x7f4eb80b27e0 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 -- 172.21.15.105:0/1402884915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80b27e0 msgr2=0x7f4eb80b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1402884915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80b27e0 0x7f4eb80b2c50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8071470 tx=0x7f4eb002bee0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 -- 172.21.15.105:0/1402884915 shutdown_connections 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1402884915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80b27e0 0x7f4eb80b2c50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1402884915 >> v2:172.21.15.132:3300/0 conn(0x7f4eb80cc7b0 0x7f4eb80ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1402884915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb814aa40 0x7f4eb80cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 -- 172.21.15.105:0/1402884915 >> 172.21.15.105:0/1402884915 conn(0x7f4eb806a3d0 msgr2=0x7f4eb80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 -- 172.21.15.105:0/1402884915 shutdown_connections 2024-03-23T07:57:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.563+0000 7f4ebde00640 1 -- 172.21.15.105:0/1402884915 wait complete. 2024-03-23T07:57:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.565+0000 7f4ebde00640 1 Processor -- start 2024-03-23T07:57:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.565+0000 7f4ebde00640 1 -- start start 2024-03-23T07:57:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.566+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb80b27e0 0x7f4eb81426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.566+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80cc7b0 0x7f4eb80a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.566+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb814aa40 0x7f4eb80a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.566+0000 7f4ebde00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81fa260 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.566+0000 7f4ebde00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81f9bc0 con 0x7f4eb814aa40 2024-03-23T07:57:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.566+0000 7f4ebde00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb8153d40 con 0x7f4eb80b27e0 2024-03-23T07:57:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.566+0000 7f4eae000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80cc7b0 0x7f4eb80a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80cc7b0 0x7f4eb80a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37642/0 (socket says 172.21.15.105:37642) 2024-03-23T07:57:35.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 -- 172.21.15.105:0/1481583677 learned_addr learned my addr 172.21.15.105:0/1481583677 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:35.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eb6a00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb80b27e0 0x7f4eb81426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eb7400640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb814aa40 0x7f4eb80a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 -- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb80b27e0 msgr2=0x7f4eb81426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb80b27e0 0x7f4eb81426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 -- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb814aa40 msgr2=0x7f4eb80a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb814aa40 0x7f4eb80a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 -- 172.21.15.105:0/1481583677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0009b30 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eae000640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80cc7b0 0x7f4eb80a74a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c7f0 tx=0x7f4ea000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eafe00640 1 -- 172.21.15.105:0/1481583677 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0007a30 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eafe00640 1 -- 172.21.15.105:0/1481583677 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea0007b90 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.567+0000 7f4eafe00640 1 -- 172.21.15.105:0/1481583677 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000ce40 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.568+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb81f9e80 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.568+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb81fa680 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.569+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb81fa3a0 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.570+0000 7f4eafe00640 1 -- 172.21.15.105:0/1481583677 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4ea00113f0 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.571+0000 7f4eafe00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e9406ac00 0x7f4e9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.571+0000 7f4eafe00640 1 -- 172.21.15.105:0/1481583677 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4ea0089870 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.571+0000 7f4eb6a00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e9406ac00 0x7f4e9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.572+0000 7f4eb6a00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e9406ac00 0x7f4e9406d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800a8d0 tx=0x7f4ea8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.576+0000 7f4eafe00640 1 -- 172.21.15.105:0/1481583677 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4eb81fa3a0 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.768+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4eb81fa3a0 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.770+0000 7f4eafe00640 1 -- 172.21.15.105:0/1481583677 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f4eb81fa3a0 con 0x7f4eb80cc7b0 2024-03-23T07:57:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:35.771 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:35.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.772+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e9406ac00 msgr2=0x7f4e9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.772+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e9406ac00 0x7f4e9406d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800a8d0 tx=0x7f4ea8008040 comp rx=0 tx=0).stop 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.772+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80cc7b0 msgr2=0x7f4eb80a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.772+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80cc7b0 0x7f4eb80a74a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c7f0 tx=0x7f4ea000cbf0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 shutdown_connections 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb814aa40 0x7f4eb80a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb80cc7b0 0x7f4eb80a74a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f4e9406ac00 0x7f4e9406d0c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 --2- 172.21.15.105:0/1481583677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4eb80b27e0 0x7f4eb81426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 >> 172.21.15.105:0/1481583677 conn(0x7f4eb806a3d0 msgr2=0x7f4eb80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 shutdown_connections 2024-03-23T07:57:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.773+0000 7f4ebde00640 1 -- 172.21.15.105:0/1481583677 wait complete. 2024-03-23T07:57:35.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:56:15.951945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:56:16.662976+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-23T07:56:20.014100+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T07:57:35.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:35.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.992+0000 7fc44d400640 1 Processor -- start 2024-03-23T07:57:35.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.992+0000 7fc44d400640 1 -- start start 2024-03-23T07:57:35.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc44d400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc448070cd0 0x7fc448160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc44d400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44806d390 con 0x7fc4481456a0 2024-03-23T07:57:35.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc44d400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44806d670 con 0x7fc4480cf040 2024-03-23T07:57:35.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc44d400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4480ad890 con 0x7fc448070cd0 2024-03-23T07:57:35.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc446800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc4481456a0 0x7fc4480ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60510/0 (socket says 172.21.15.105:60510) 2024-03-23T07:57:35.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc446800640 1 -- 172.21.15.105:0/2232850697 learned_addr learned my addr 172.21.15.105:0/2232850697 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022606782 0 0) 0x7fc44806d390 con 0x7fc4481456a0 2024-03-23T07:57:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc447200640 1 --2- 172.21.15.105:0/2232850697 >> v2:172.21.15.132:3300/0 conn(0x7fc448070cd0 0x7fc448160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.994+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc42c003520 con 0x7fc4481456a0 2024-03-23T07:57:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3845635255 0 0) 0x7fc42c003520 con 0x7fc4481456a0 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 >> v2:172.21.15.132:3300/0 conn(0x7fc448070cd0 msgr2=0x7fc448160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 --2- 172.21.15.105:0/2232850697 >> v2:172.21.15.132:3300/0 conn(0x7fc448070cd0 0x7fc448160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 >> v1:172.21.15.132:6789/0 conn(0x7fc4480cf040 legacy=0x7fc4481604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480701f0 con 0x7fc4481456a0 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fc438001670 con 0x7fc4481456a0 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 >> v1:172.21.15.105:6789/0 conn(0x7fc4481456a0 legacy=0x7fc4480ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c003970 0x7fc42c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42c0047c0 0x7fc42c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c005180 0x7fc42c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480ad890 con 0x7fc42c005180 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c00c0b0 con 0x7fc42c0047c0 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c00c270 con 0x7fc42c003970 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc447200640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c005180 0x7fc42c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc446800640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c003970 0x7fc42c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc447200640 1 -- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c003970 msgr2=0x7fc42c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc447200640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c003970 0x7fc42c003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc447200640 1 -- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42c0047c0 msgr2=0x7fc42c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.995+0000 7fc43fe00640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42c0047c0 0x7fc42c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.996+0000 7fc447200640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42c0047c0 0x7fc42c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.996+0000 7fc447200640 1 -- 172.21.15.105:0/2232850697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c003520 con 0x7fc42c005180 2024-03-23T07:57:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.996+0000 7fc43fe00640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42c0047c0 0x7fc42c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:35.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.996+0000 7fc447200640 1 -- 172.21.15.105:0/2232850697 reap_dead start 2024-03-23T07:57:35.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc447200640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c005180 0x7fc42c00b980 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc440007330 tx=0x7fc42c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5219d40ef326e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:35.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44002ccd0 con 0x7fc42c005180 2024-03-23T07:57:35.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44002ce30 con 0x7fc42c005180 2024-03-23T07:57:35.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc445e00640 1 -- 172.21.15.105:0/2232850697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440003bd0 con 0x7fc42c005180 2024-03-23T07:57:35.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc44d400640 1 -- 172.21.15.105:0/2232850697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c005180 msgr2=0x7fc42c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:35.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc44d400640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c005180 0x7fc42c00b980 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc440007330 tx=0x7fc42c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc44d400640 1 -- 172.21.15.105:0/2232850697 shutdown_connections 2024-03-23T07:57:36.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc44d400640 1 --2- 172.21.15.105:0/2232850697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c005180 0x7fc42c00b980 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.997+0000 7fc44d400640 1 -- 172.21.15.105:0/2232850697 >> 172.21.15.105:0/2232850697 conn(0x7fc44806a0b0 msgr2=0x7fc44806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:36.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.998+0000 7fc44d400640 1 -- 172.21.15.105:0/2232850697 shutdown_connections 2024-03-23T07:57:36.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.998+0000 7fc44d400640 1 -- 172.21.15.105:0/2232850697 wait complete. 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.999+0000 7fc44d400640 1 Processor -- start 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:35.999+0000 7fc44d400640 1 -- start start 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.000+0000 7fc44d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c005180 0x7fc44815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.000+0000 7fc44d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480c3530 0x7fc4480c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.000+0000 7fc44d400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4481f53a0 0x7fc4481f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.000+0000 7fc44d400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480b0200 con 0x7fc4480c3530 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.000+0000 7fc44d400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480c3ef0 con 0x7fc4481f53a0 2024-03-23T07:57:36.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.000+0000 7fc44d400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4481f5d90 con 0x7fc42c005180 2024-03-23T07:57:36.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480c3530 0x7fc4480c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480c3530 0x7fc4480c39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37656/0 (socket says 172.21.15.105:37656) 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 -- 172.21.15.105:0/1640755450 learned_addr learned my addr 172.21.15.105:0/1640755450 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc446800640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c005180 0x7fc44815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc447200640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4481f53a0 0x7fc4481f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 -- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c005180 msgr2=0x7fc44815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c005180 0x7fc44815bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 -- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4481f53a0 msgr2=0x7fc4481f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4481f53a0 0x7fc4481f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 -- 172.21.15.105:0/1640755450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4481f6290 con 0x7fc4480c3530 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.001+0000 7fc43fe00640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480c3530 0x7fc4480c39b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc43000d590 tx=0x7fc43000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.002+0000 7fc43ea00640 1 -- 172.21.15.105:0/1640755450 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430007220 con 0x7fc4480c3530 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.002+0000 7fc43ea00640 1 -- 172.21.15.105:0/1640755450 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300073a0 con 0x7fc4480c3530 2024-03-23T07:57:36.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.002+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44815c140 con 0x7fc4480c3530 2024-03-23T07:57:36.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.002+0000 7fc43ea00640 1 -- 172.21.15.105:0/1640755450 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300067d0 con 0x7fc4480c3530 2024-03-23T07:57:36.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.002+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4481f3010 con 0x7fc4480c3530 2024-03-23T07:57:36.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.003+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc448073390 con 0x7fc4480c3530 2024-03-23T07:57:36.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.004+0000 7fc43ea00640 1 -- 172.21.15.105:0/1640755450 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc430006930 con 0x7fc4480c3530 2024-03-23T07:57:36.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.005+0000 7fc43ea00640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc42406ac30 0x7fc42406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.005+0000 7fc43ea00640 1 -- 172.21.15.105:0/1640755450 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc430089410 con 0x7fc4480c3530 2024-03-23T07:57:36.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.005+0000 7fc446800640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc42406ac30 0x7fc42406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.006+0000 7fc446800640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc42406ac30 0x7fc42406d0f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4380017a0 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.009+0000 7fc43ea00640 1 -- 172.21.15.105:0/1640755450 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc430054200 con 0x7fc4480c3530 2024-03-23T07:57:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.197+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc44806cbc0 con 0x7fc4480c3530 2024-03-23T07:57:36.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.198+0000 7fc43ea00640 1 -- 172.21.15.105:0/1640755450 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc430057820 con 0x7fc4480c3530 2024-03-23T07:57:36.198 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc42406ac30 msgr2=0x7fc42406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc42406ac30 0x7fc42406d0f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4380017a0 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-03-23T07:57:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480c3530 msgr2=0x7fc4480c39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480c3530 0x7fc4480c39b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc43000d590 tx=0x7fc43000da50 comp rx=0 tx=0).stop 2024-03-23T07:57:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 shutdown_connections 2024-03-23T07:57:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4481f53a0 0x7fc4481f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480c3530 0x7fc4480c39b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc42406ac30 0x7fc42406d0f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 --2- 172.21.15.105:0/1640755450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42c005180 0x7fc44815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 >> 172.21.15.105:0/1640755450 conn(0x7fc44806a0b0 msgr2=0x7fc4480b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 shutdown_connections 2024-03-23T07:57:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.200+0000 7fc44d400640 1 -- 172.21.15.105:0/1640755450 wait complete. 2024-03-23T07:57:36.219 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:57:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.416+0000 7f0fb6200640 1 Processor -- start 2024-03-23T07:57:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.416+0000 7f0fb6200640 1 -- start start 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0fb6200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b84b0 0x7f0fb00d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0fb6200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0fb00b5aa0 0x7f0fb00b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0fb6200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb00d1280 con 0x7f0fb00cd740 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0fb6200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb00d20c0 con 0x7f0fb00b5aa0 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0fb6200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b8910 con 0x7f0fb00b84b0 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0faf400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0fb00cd740 0x7f0fb00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60522/0 (socket says 172.21.15.105:60522) 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0faf400640 1 -- 172.21.15.105:0/4107666546 learned_addr learned my addr 172.21.15.105:0/4107666546 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.418+0000 7f0fafe00640 1 --2- 172.21.15.105:0/4107666546 >> v2:172.21.15.105:3300/0 conn(0x7f0fb00b5aa0 0x7f0fb00b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.419+0000 7f0faea00640 1 --2- 172.21.15.105:0/4107666546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b84b0 0x7f0fb00d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.419+0000 7f0fae000640 1 -- 172.21.15.105:0/4107666546 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565470238 0 0) 0x7f0fb00d1280 con 0x7f0fb00cd740 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.419+0000 7f0fafe00640 1 -- 172.21.15.105:0/4107666546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b84b0 msgr2=0x7f0fb00d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.419+0000 7f0fafe00640 1 --2- 172.21.15.105:0/4107666546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b84b0 0x7f0fb00d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.419+0000 7f0fafe00640 1 -- 172.21.15.105:0/4107666546 >> v1:172.21.15.105:6789/0 conn(0x7f0fb00cd740 legacy=0x7f0fb00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.419+0000 7f0fafe00640 1 -- 172.21.15.105:0/4107666546 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00d14b0 con 0x7f0fb00b5aa0 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.419+0000 7f0faea00640 1 --2- 172.21.15.105:0/4107666546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b84b0 0x7f0fb00d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.420+0000 7f0fafe00640 1 --2- 172.21.15.105:0/4107666546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b5aa0 0x7f0fb00b5f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80097a0 tx=0x7f0fa802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37354a50ba6faa58 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.420+0000 7f0fae000640 1 -- 172.21.15.105:0/4107666546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa802d040 con 0x7f0fb00b5aa0 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.420+0000 7f0fae000640 1 -- 172.21.15.105:0/4107666546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa8031810 con 0x7f0fb00b5aa0 2024-03-23T07:57:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.420+0000 7f0fae000640 1 -- 172.21.15.105:0/4107666546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8031ba0 con 0x7f0fb00b5aa0 2024-03-23T07:57:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 -- 172.21.15.105:0/4107666546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b5aa0 msgr2=0x7f0fb00b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 --2- 172.21.15.105:0/4107666546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b5aa0 0x7f0fb00b5f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80097a0 tx=0x7f0fa802b8b0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 -- 172.21.15.105:0/4107666546 shutdown_connections 2024-03-23T07:57:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 --2- 172.21.15.105:0/4107666546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b5aa0 0x7f0fb00b5f30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 --2- 172.21.15.105:0/4107666546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b84b0 0x7f0fb00d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 -- 172.21.15.105:0/4107666546 >> 172.21.15.105:0/4107666546 conn(0x7f0fb00cf5d0 msgr2=0x7f0fb00cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 -- 172.21.15.105:0/4107666546 shutdown_connections 2024-03-23T07:57:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.421+0000 7f0fb6200640 1 -- 172.21.15.105:0/4107666546 wait complete. 2024-03-23T07:57:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.423+0000 7f0fb6200640 1 Processor -- start 2024-03-23T07:57:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.423+0000 7f0fb6200640 1 -- start start 2024-03-23T07:57:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0fb6200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b5aa0 0x7f0fb01566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0fb6200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b84b0 0x7f0fb0152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0fb6200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb00cd740 0x7f0fb0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0fb6200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b3320 con 0x7f0fb00b84b0 2024-03-23T07:57:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0fb6200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b3600 con 0x7f0fb00cd740 2024-03-23T07:57:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0fb6200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b38e0 con 0x7f0fb00b5aa0 2024-03-23T07:57:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b84b0 0x7f0fb0152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b84b0 0x7f0fb0152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37668/0 (socket says 172.21.15.105:37668) 2024-03-23T07:57:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 -- 172.21.15.105:0/133933212 learned_addr learned my addr 172.21.15.105:0/133933212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 -- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b5aa0 msgr2=0x7f0fb01566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faf400640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b5aa0 0x7f0fb01566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b5aa0 0x7f0fb01566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0fafe00640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb00cd740 0x7f0fb0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 -- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb00cd740 msgr2=0x7f0fb0154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb00cd740 0x7f0fb0154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.424+0000 7f0faea00640 1 -- 172.21.15.105:0/133933212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa8009400 con 0x7f0fb00b84b0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.425+0000 7f0faea00640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b84b0 0x7f0fb0152ce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800cf60 tx=0x7f0f9800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.425+0000 7f0fa7400640 1 -- 172.21.15.105:0/133933212 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800d840 con 0x7f0fb00b84b0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.425+0000 7f0fa7400640 1 -- 172.21.15.105:0/133933212 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9800d9a0 con 0x7f0fb00b84b0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.425+0000 7f0fa7400640 1 -- 172.21.15.105:0/133933212 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800dd30 con 0x7f0fb00b84b0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.425+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb00b3bc0 con 0x7f0fb00b84b0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.425+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb0153220 con 0x7f0fb00b84b0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.426+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb00b40e0 con 0x7f0fb00b84b0 2024-03-23T07:57:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.427+0000 7f0fa7400640 1 -- 172.21.15.105:0/133933212 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0f98015440 con 0x7f0fb00b84b0 2024-03-23T07:57:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.428+0000 7f0fa7400640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f8c06ac00 0x7f0f8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.428+0000 7f0fa7400640 1 -- 172.21.15.105:0/133933212 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0f9808a0e0 con 0x7f0fb00b84b0 2024-03-23T07:57:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.428+0000 7f0faf400640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f8c06ac00 0x7f0f8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.429+0000 7f0faf400640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f8c06ac00 0x7f0f8c06d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000f60 tx=0x7f0fa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.432+0000 7f0fa7400640 1 -- 172.21.15.105:0/133933212 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0fb00b40e0 con 0x7f0fb00b84b0 2024-03-23T07:57:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.629+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0fb00b40e0 con 0x7f0fb00b84b0 2024-03-23T07:57:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.629+0000 7f0fa7400640 1 -- 172.21.15.105:0/133933212 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0fb00b40e0 con 0x7f0fb00b84b0 2024-03-23T07:57:36.630 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f8c06ac00 msgr2=0x7f0f8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f8c06ac00 0x7f0f8c06d0c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000f60 tx=0x7f0fa0009040 comp rx=0 tx=0).stop 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b84b0 msgr2=0x7f0fb0152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b84b0 0x7f0fb0152ce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800cf60 tx=0x7f0f9800cf90 comp rx=0 tx=0).stop 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 shutdown_connections 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb00cd740 0x7f0fb0154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fb00b84b0 0x7f0fb0152ce0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0f8c06ac00 0x7f0f8c06d0c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 --2- 172.21.15.105:0/133933212 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fb00b5aa0 0x7f0fb01566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 >> 172.21.15.105:0/133933212 conn(0x7f0fb00cf5d0 msgr2=0x7f0fb00b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 shutdown_connections 2024-03-23T07:57:36.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.632+0000 7f0fb6200640 1 -- 172.21.15.105:0/133933212 wait complete. 2024-03-23T07:57:36.653 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:57:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.865+0000 7f0b62000640 1 Processor -- start 2024-03-23T07:57:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.866+0000 7f0b62000640 1 -- start start 2024-03-23T07:57:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.867+0000 7f0b62000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c0add30 0x7f0b5c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.867+0000 7f0b62000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0b5c06ce90 0x7f0b5c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.867+0000 7f0b62000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5c0cf250 con 0x7f0b5c0ce930 2024-03-23T07:57:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.867+0000 7f0b62000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5c06c760 con 0x7f0b5c06ce90 2024-03-23T07:57:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.867+0000 7f0b62000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0b42d0 con 0x7f0b5c0add30 2024-03-23T07:57:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.867+0000 7f0b5b400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0b5c0ce930 0x7f0b5c0cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55972/0 (socket says 172.21.15.105:55972) 2024-03-23T07:57:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.867+0000 7f0b5b400640 1 -- 172.21.15.105:0/3921468712 learned_addr learned my addr 172.21.15.105:0/3921468712 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b52a00640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c0add30 0x7f0b5c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5be00640 1 --2- 172.21.15.105:0/3921468712 >> v2:172.21.15.132:3300/0 conn(0x7f0b5c06ce90 0x7f0b5c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67377658 0 0) 0x7f0b5c0cf250 con 0x7f0b5c0ce930 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b40003520 con 0x7f0b5c0ce930 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2032240825 0 0) 0x7f0b40003520 con 0x7f0b5c0ce930 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c0add30 msgr2=0x7f0b5c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c0add30 0x7f0b5c0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 >> v2:172.21.15.132:3300/0 conn(0x7f0b5c06ce90 msgr2=0x7f0b5c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 --2- 172.21.15.105:0/3921468712 >> v2:172.21.15.132:3300/0 conn(0x7f0b5c06ce90 0x7f0b5c06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0cdba0 con 0x7f0b5c0ce930 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f0b4c001670 con 0x7f0b5c0ce930 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 >> v1:172.21.15.105:6789/0 conn(0x7f0b5c0ce930 legacy=0x7f0b5c0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b40003970 0x7f0b40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b400047c0 0x7f0b40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b40005180 0x7f0b4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c06c760 con 0x7f0b40003970 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0b42d0 con 0x7f0b400047c0 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4000c020 con 0x7f0b40005180 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b5be00640 1 --2- 172.21.15.105:0/3921468712 >> v2:172.21.15.132:3300/0 conn(0x7f0b5c06ce90 0x7f0b5c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.868+0000 7f0b52a00640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c0add30 0x7f0b5c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.869+0000 7f0b5b400640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b40003970 0x7f0b40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.869+0000 7f0b5b400640 1 -- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b40005180 msgr2=0x7f0b4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.869+0000 7f0b5b400640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b40005180 0x7f0b4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.869+0000 7f0b5b400640 1 -- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b400047c0 msgr2=0x7f0b40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.869+0000 7f0b5b400640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b400047c0 0x7f0b40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.869+0000 7f0b5b400640 1 -- 172.21.15.105:0/3921468712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40003520 con 0x7f0b40003970 2024-03-23T07:57:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.870+0000 7f0b5b400640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b40003970 0x7f0b40003d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00c110 tx=0x7f0b400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91f5dc3700fe646 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.870+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c005ad0 con 0x7f0b40003970 2024-03-23T07:57:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.870+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c005c50 con 0x7f0b40003970 2024-03-23T07:57:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.870+0000 7f0b5aa00640 1 -- 172.21.15.105:0/3921468712 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c02c830 con 0x7f0b40003970 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.871+0000 7f0b62000640 1 -- 172.21.15.105:0/3921468712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b40003970 msgr2=0x7f0b40003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.871+0000 7f0b62000640 1 --2- 172.21.15.105:0/3921468712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b40003970 0x7f0b40003d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00c110 tx=0x7f0b400057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.871+0000 7f0b5be00640 1 -- 172.21.15.105:0/3921468712 reap_dead start 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.871+0000 7f0b62000640 1 -- 172.21.15.105:0/3921468712 shutdown_connections 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.871+0000 7f0b62000640 1 -- 172.21.15.105:0/3921468712 >> 172.21.15.105:0/3921468712 conn(0x7f0b5c06a0b0 msgr2=0x7f0b5c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.871+0000 7f0b62000640 1 -- 172.21.15.105:0/3921468712 shutdown_connections 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.871+0000 7f0b62000640 1 -- 172.21.15.105:0/3921468712 wait complete. 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.873+0000 7f0b62000640 1 Processor -- start 2024-03-23T07:57:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.873+0000 7f0b62000640 1 -- start start 2024-03-23T07:57:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.874+0000 7f0b62000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b40003970 0x7f0b5c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.874+0000 7f0b62000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b5c150260 0x7f0b5c1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b62000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c14ada0 0x7f0b5c14b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b62000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c14c140 con 0x7f0b5c150260 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b62000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c150c20 con 0x7f0b40003970 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b62000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c14e2d0 con 0x7f0b5c14ada0 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b5c150260 0x7f0b5c1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b5c150260 0x7f0b5c1506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52152/0 (socket says 172.21.15.105:52152) 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 -- 172.21.15.105:0/2211967122 learned_addr learned my addr 172.21.15.105:0/2211967122 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 -- 172.21.15.105:0/2211967122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c14ada0 msgr2=0x7f0b5c14b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c14ada0 0x7f0b5c14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 -- 172.21.15.105:0/2211967122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b40003970 msgr2=0x7f0b5c1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b40003970 0x7f0b5c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.875+0000 7f0b52a00640 1 -- 172.21.15.105:0/2211967122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c14b760 con 0x7f0b5c150260 2024-03-23T07:57:36.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.876+0000 7f0b52a00640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b5c150260 0x7f0b5c1506e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400e560 tx=0x7f0b4400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.876+0000 7f0b53e00640 1 -- 172.21.15.105:0/2211967122 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44004230 con 0x7f0b5c150260 2024-03-23T07:57:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.876+0000 7f0b53e00640 1 -- 172.21.15.105:0/2211967122 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44004390 con 0x7f0b5c150260 2024-03-23T07:57:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.876+0000 7f0b53e00640 1 -- 172.21.15.105:0/2211967122 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44003830 con 0x7f0b5c150260 2024-03-23T07:57:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.876+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c14e540 con 0x7f0b5c150260 2024-03-23T07:57:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.876+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c14eb00 con 0x7f0b5c150260 2024-03-23T07:57:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.877+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c073390 con 0x7f0b5c150260 2024-03-23T07:57:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.878+0000 7f0b53e00640 1 -- 172.21.15.105:0/2211967122 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b44003990 con 0x7f0b5c150260 2024-03-23T07:57:36.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.879+0000 7f0b53e00640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b3806ac00 0x7f0b3806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:36.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.879+0000 7f0b53e00640 1 -- 172.21.15.105:0/2211967122 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0b44089110 con 0x7f0b5c150260 2024-03-23T07:57:36.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.880+0000 7f0b5b400640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b3806ac00 0x7f0b3806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:36.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.880+0000 7f0b5b400640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b3806ac00 0x7f0b3806d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c034000 tx=0x7f0b4c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:36.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:36.885+0000 7f0b53e00640 1 -- 172.21.15.105:0/2211967122 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b44054030 con 0x7f0b5c150260 2024-03-23T07:57:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.078+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b5c06c540 con 0x7f0b5c150260 2024-03-23T07:57:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.079+0000 7f0b53e00640 1 -- 172.21.15.105:0/2211967122 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b44057650 con 0x7f0b5c150260 2024-03-23T07:57:37.079 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:57:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b3806ac00 msgr2=0x7f0b3806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b3806ac00 0x7f0b3806d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c034000 tx=0x7f0b4c02e000 comp rx=0 tx=0).stop 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b5c150260 msgr2=0x7f0b5c1506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b5c150260 0x7f0b5c1506e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400e560 tx=0x7f0b4400ea20 comp rx=0 tx=0).stop 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 shutdown_connections 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b3806ac00 0x7f0b3806d0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b5c14ada0 0x7f0b5c14b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b5c150260 0x7f0b5c1506e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 --2- 172.21.15.105:0/2211967122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b40003970 0x7f0b5c1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.081+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 >> 172.21.15.105:0/2211967122 conn(0x7f0b5c06a0b0 msgr2=0x7f0b5c0b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.082+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 shutdown_connections 2024-03-23T07:57:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.082+0000 7f0b62000640 1 -- 172.21.15.105:0/2211967122 wait complete. 2024-03-23T07:57:37.100 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-23T07:57:37.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.316+0000 7fda69600640 1 Processor -- start 2024-03-23T07:57:37.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.316+0000 7fda69600640 1 -- start start 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda69600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fda640cf7c0 0x7fda640cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda69600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda640ccd90 con 0x7fda6415c820 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda69600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda640b7fc0 con 0x7fda64160e90 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda69600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda640cdd70 con 0x7fda640cf7c0 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda63e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fda6415c820 0x7fda640cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55982/0 (socket says 172.21.15.105:55982) 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda63e00640 1 -- 172.21.15.105:0/2510565849 learned_addr learned my addr 172.21.15.105:0/2510565849 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda63400640 1 --2- 172.21.15.105:0/2510565849 >> v2:172.21.15.132:3300/0 conn(0x7fda640cf7c0 0x7fda640cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.318+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397849128 0 0) 0x7fda640ccd90 con 0x7fda6415c820 2024-03-23T07:57:37.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda48003520 con 0x7fda6415c820 2024-03-23T07:57:37.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3001193277 0 0) 0x7fda48003520 con 0x7fda6415c820 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 >> v2:172.21.15.132:3300/0 conn(0x7fda640cf7c0 msgr2=0x7fda640cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 --2- 172.21.15.105:0/2510565849 >> v2:172.21.15.132:3300/0 conn(0x7fda640cf7c0 0x7fda640cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 >> v1:172.21.15.132:6789/0 conn(0x7fda64160e90 legacy=0x7fda640d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda640b82a0 con 0x7fda6415c820 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fda54001670 con 0x7fda6415c820 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 >> v1:172.21.15.105:6789/0 conn(0x7fda6415c820 legacy=0x7fda640cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda480039c0 0x7fda48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda480047c0 0x7fda48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480051d0 0x7fda4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640cdd70 con 0x7fda480039c0 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4800c0e0 con 0x7fda480051d0 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4800c2a0 con 0x7fda480047c0 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.319+0000 7fda63e00640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda480039c0 0x7fda48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.320+0000 7fda63400640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda480047c0 0x7fda48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.320+0000 7fda63e00640 1 -- 172.21.15.105:0/2510565849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda480047c0 msgr2=0x7fda48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.320+0000 7fda63e00640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda480047c0 0x7fda48004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.320+0000 7fda63e00640 1 -- 172.21.15.105:0/2510565849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480051d0 msgr2=0x7fda4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.320+0000 7fda63e00640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480051d0 0x7fda4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.320+0000 7fda63e00640 1 -- 172.21.15.105:0/2510565849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda48003520 con 0x7fda480039c0 2024-03-23T07:57:37.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.320+0000 7fda68c00640 1 -- 172.21.15.105:0/2510565849 reap_dead start 2024-03-23T07:57:37.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.321+0000 7fda63e00640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda480039c0 0x7fda48003dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fda5400c190 tx=0x7fda480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7618e89ee53bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:37.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.321+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54005ad0 con 0x7fda480039c0 2024-03-23T07:57:37.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.321+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda54005c30 con 0x7fda480039c0 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.321+0000 7fda62a00640 1 -- 172.21.15.105:0/2510565849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5402c810 con 0x7fda480039c0 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.321+0000 7fda69600640 1 -- 172.21.15.105:0/2510565849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda480039c0 msgr2=0x7fda48003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.321+0000 7fda69600640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda480039c0 0x7fda48003dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fda5400c190 tx=0x7fda480057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.322+0000 7fda69600640 1 -- 172.21.15.105:0/2510565849 shutdown_connections 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.322+0000 7fda69600640 1 --2- 172.21.15.105:0/2510565849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda480039c0 0x7fda48003dc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.322+0000 7fda69600640 1 -- 172.21.15.105:0/2510565849 >> 172.21.15.105:0/2510565849 conn(0x7fda6406b050 msgr2=0x7fda6406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.322+0000 7fda69600640 1 -- 172.21.15.105:0/2510565849 shutdown_connections 2024-03-23T07:57:37.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.322+0000 7fda69600640 1 -- 172.21.15.105:0/2510565849 wait complete. 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.323+0000 7fda69600640 1 Processor -- start 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.323+0000 7fda69600640 1 -- start start 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.324+0000 7fda69600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480039c0 0x7fda64144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.324+0000 7fda69600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda64158e80 0x7fda6415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.324+0000 7fda69600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6415f2d0 0x7fda641f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.324+0000 7fda69600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6415e900 con 0x7fda6415f2d0 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.324+0000 7fda69600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6415dcf0 con 0x7fda480039c0 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.324+0000 7fda69600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6415dfd0 con 0x7fda64158e80 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6415f2d0 0x7fda641f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6415f2d0 0x7fda641f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52180/0 (socket says 172.21.15.105:52180) 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 -- 172.21.15.105:0/1831003776 learned_addr learned my addr 172.21.15.105:0/1831003776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda63400640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda64158e80 0x7fda6415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda63e00640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480039c0 0x7fda64144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 -- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda64158e80 msgr2=0x7fda6415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda64158e80 0x7fda6415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 -- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480039c0 msgr2=0x7fda64144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480039c0 0x7fda64144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.325+0000 7fda68c00640 1 -- 172.21.15.105:0/1831003776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6415e2b0 con 0x7fda6415f2d0 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.326+0000 7fda68c00640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6415f2d0 0x7fda641f5570 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fda640ce4d0 tx=0x7fda5c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.326+0000 7fda5be00640 1 -- 172.21.15.105:0/1831003776 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c007070 con 0x7fda6415f2d0 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.326+0000 7fda5be00640 1 -- 172.21.15.105:0/1831003776 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c0071d0 con 0x7fda6415f2d0 2024-03-23T07:57:37.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.326+0000 7fda5be00640 1 -- 172.21.15.105:0/1831003776 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c007580 con 0x7fda6415f2d0 2024-03-23T07:57:37.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.326+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6415e590 con 0x7fda6415f2d0 2024-03-23T07:57:37.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.326+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda641594c0 con 0x7fda6415f2d0 2024-03-23T07:57:37.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.327+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6404e030 con 0x7fda6415f2d0 2024-03-23T07:57:37.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.328+0000 7fda5be00640 1 -- 172.21.15.105:0/1831003776 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fda5c0076e0 con 0x7fda6415f2d0 2024-03-23T07:57:37.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.329+0000 7fda5be00640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fda4406ac00 0x7fda4406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.329+0000 7fda5be00640 1 -- 172.21.15.105:0/1831003776 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fda5c0889f0 con 0x7fda6415f2d0 2024-03-23T07:57:37.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.329+0000 7fda63e00640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fda4406ac00 0x7fda4406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.330+0000 7fda63e00640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fda4406ac00 0x7fda4406d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda54034000 tx=0x7fda54009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:37.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.333+0000 7fda5be00640 1 -- 172.21.15.105:0/1831003776 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fda5c0537e0 con 0x7fda6415f2d0 2024-03-23T07:57:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.527+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fda641594c0 con 0x7fda6415f2d0 2024-03-23T07:57:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.528+0000 7fda5be00640 1 -- 172.21.15.105:0/1831003776 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda641594c0 con 0x7fda6415f2d0 2024-03-23T07:57:37.528 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:57:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.530+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fda4406ac00 msgr2=0x7fda4406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.530+0000 7fda69600640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fda4406ac00 0x7fda4406d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda54034000 tx=0x7fda54009030 comp rx=0 tx=0).stop 2024-03-23T07:57:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.530+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6415f2d0 msgr2=0x7fda641f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.530+0000 7fda69600640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6415f2d0 0x7fda641f5570 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fda640ce4d0 tx=0x7fda5c00c960 comp rx=0 tx=0).stop 2024-03-23T07:57:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 shutdown_connections 2024-03-23T07:57:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6415f2d0 0x7fda641f5570 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fda4406ac00 0x7fda4406d0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda64158e80 0x7fda6415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 --2- 172.21.15.105:0/1831003776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda480039c0 0x7fda64144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 >> 172.21.15.105:0/1831003776 conn(0x7fda6406b050 msgr2=0x7fda640b6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 shutdown_connections 2024-03-23T07:57:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.531+0000 7fda69600640 1 -- 172.21.15.105:0/1831003776 wait complete. 2024-03-23T07:57:37.550 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:57:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.751+0000 7fe6f7200640 1 Processor -- start 2024-03-23T07:57:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.751+0000 7fe6f7200640 1 -- start start 2024-03-23T07:57:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.752+0000 7fe6f7200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6f8143460 0x7fe6f815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.752+0000 7fe6f7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6f80cde20 0x7fe6f8070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.752+0000 7fe6f7200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f80cc6b0 con 0x7fe6f8143930 2024-03-23T07:57:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.752+0000 7fe6f7200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80cc950 con 0x7fe6f8143460 2024-03-23T07:57:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.752+0000 7fe6f7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80ccc30 con 0x7fe6f80cde20 2024-03-23T07:57:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f5400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe6f8143930 0x7fe6f80cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55984/0 (socket says 172.21.15.105:55984) 2024-03-23T07:57:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f5400640 1 -- 172.21.15.105:0/3882867971 learned_addr learned my addr 172.21.15.105:0/3882867971 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f5e00640 1 --2- 172.21.15.105:0/3882867971 >> v2:172.21.15.132:3300/0 conn(0x7fe6f8143460 0x7fe6f815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f6800640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6f80cde20 0x7fe6f8070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527596586 0 0) 0x7fe6f80cc6b0 con 0x7fe6f8143930 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6dc003520 con 0x7fe6f8143930 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 807042679 0 0) 0x7fe6dc003520 con 0x7fe6f8143930 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6f80cde20 msgr2=0x7fe6f8070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.753+0000 7fe6f4a00640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6f80cde20 0x7fe6f8070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 >> v2:172.21.15.132:3300/0 conn(0x7fe6f8143460 msgr2=0x7fe6f815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 --2- 172.21.15.105:0/3882867971 >> v2:172.21.15.132:3300/0 conn(0x7fe6f8143460 0x7fe6f815aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80ccf10 con 0x7fe6f8143930 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fe6e4001670 con 0x7fe6f8143930 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 >> v1:172.21.15.105:6789/0 conn(0x7fe6f8143930 legacy=0x7fe6f80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6dc003970 0x7fe6dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc0047c0 0x7fe6dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6dc005180 0x7fe6dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80cc950 con 0x7fe6dc005180 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80ccc30 con 0x7fe6dc003970 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc00bfa0 con 0x7fe6dc0047c0 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f6800640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6f80cde20 0x7fe6f8070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f5e00640 1 --2- 172.21.15.105:0/3882867971 >> v2:172.21.15.132:3300/0 conn(0x7fe6f8143460 0x7fe6f815aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f6800640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6dc005180 0x7fe6dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f5e00640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6dc003970 0x7fe6dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.754+0000 7fe6f5400640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc0047c0 0x7fe6dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.755+0000 7fe6f6800640 1 -- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc0047c0 msgr2=0x7fe6dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.755+0000 7fe6f6800640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc0047c0 0x7fe6dc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.755+0000 7fe6f6800640 1 -- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6dc003970 msgr2=0x7fe6dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.755+0000 7fe6f6800640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6dc003970 0x7fe6dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.755+0000 7fe6f6800640 1 -- 172.21.15.105:0/3882867971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc003520 con 0x7fe6dc005180 2024-03-23T07:57:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.755+0000 7fe6f5e00640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6dc003970 0x7fe6dc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.756+0000 7fe6f6800640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6dc005180 0x7fe6dc00b980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80ce200 tx=0x7fe6dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c245413198861c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.756+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f002cc80 con 0x7fe6dc005180 2024-03-23T07:57:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.756+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f002cde0 con 0x7fe6dc005180 2024-03-23T07:57:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.756+0000 7fe6f4a00640 1 -- 172.21.15.105:0/3882867971 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0003bb0 con 0x7fe6dc005180 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.756+0000 7fe6f7200640 1 -- 172.21.15.105:0/3882867971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6dc005180 msgr2=0x7fe6dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.756+0000 7fe6f7200640 1 --2- 172.21.15.105:0/3882867971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6dc005180 0x7fe6dc00b980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80ce200 tx=0x7fe6dc0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.756+0000 7fe6f6800640 1 -- 172.21.15.105:0/3882867971 reap_dead start 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.757+0000 7fe6f7200640 1 -- 172.21.15.105:0/3882867971 shutdown_connections 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.757+0000 7fe6f7200640 1 -- 172.21.15.105:0/3882867971 >> 172.21.15.105:0/3882867971 conn(0x7fe6f8065df0 msgr2=0x7fe6f8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.757+0000 7fe6f7200640 1 -- 172.21.15.105:0/3882867971 shutdown_connections 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.757+0000 7fe6f7200640 1 -- 172.21.15.105:0/3882867971 wait complete. 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.758+0000 7fe6f7200640 1 Processor -- start 2024-03-23T07:57:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.758+0000 7fe6f7200640 1 -- start start 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.759+0000 7fe6f7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc005180 0x7fe6f81f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.759+0000 7fe6f7200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6f81f5870 0x7fe6f806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.759+0000 7fe6f7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6f81413d0 0x7fe6f8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.759+0000 7fe6f7200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81f2830 con 0x7fe6f81f5870 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.759+0000 7fe6f7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c77f0 con 0x7fe6f81413d0 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.759+0000 7fe6f7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81f5de0 con 0x7fe6dc005180 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6f81f5870 0x7fe6f806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6f81f5870 0x7fe6f806daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52192/0 (socket says 172.21.15.105:52192) 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 -- 172.21.15.105:0/4153555715 learned_addr learned my addr 172.21.15.105:0/4153555715 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 -- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc005180 msgr2=0x7fe6f81f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5e00640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc005180 0x7fe6f81f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc005180 0x7fe6f81f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 -- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6f81413d0 msgr2=0x7fe6f8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f6800640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6f81413d0 0x7fe6f8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6f81413d0 0x7fe6f8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 -- 172.21.15.105:0/4153555715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f81f60c0 con 0x7fe6f81f5870 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f6800640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6f81413d0 0x7fe6f8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.760+0000 7fe6f5400640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6f81f5870 0x7fe6f806daf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40018a0 tx=0x7fe6e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.761+0000 7fe6ee000640 1 -- 172.21.15.105:0/4153555715 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40074e0 con 0x7fe6f81f5870 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.761+0000 7fe6ee000640 1 -- 172.21.15.105:0/4153555715 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e4007640 con 0x7fe6f81f5870 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.761+0000 7fe6ee000640 1 -- 172.21.15.105:0/4153555715 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40049e0 con 0x7fe6f81f5870 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.761+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f81f63a0 con 0x7fe6f81f5870 2024-03-23T07:57:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.761+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f806e030 con 0x7fe6f81f5870 2024-03-23T07:57:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.762+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f80c79b0 con 0x7fe6f81f5870 2024-03-23T07:57:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.763+0000 7fe6ee000640 1 -- 172.21.15.105:0/4153555715 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe6e4004b40 con 0x7fe6f81f5870 2024-03-23T07:57:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.764+0000 7fe6ee000640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe6d806ac30 0x7fe6d806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.764+0000 7fe6ee000640 1 -- 172.21.15.105:0/4153555715 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe6e4088de0 con 0x7fe6f81f5870 2024-03-23T07:57:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.764+0000 7fe6f5e00640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe6d806ac30 0x7fe6d806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.765+0000 7fe6f5e00640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe6d806ac30 0x7fe6d806d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8006fd0 tx=0x7fe6e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:37.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:37.768+0000 7fe6ee000640 1 -- 172.21.15.105:0/4153555715 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe6e4053d10 con 0x7fe6f81f5870 2024-03-23T07:57:38.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.001+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6f804e030 con 0x7fe6f81f5870 2024-03-23T07:57:38.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.002+0000 7fe6ee000640 1 -- 172.21.15.105:0/4153555715 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7fe6e4019020 con 0x7fe6f81f5870 2024-03-23T07:57:38.003 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:38.003 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:57:38.003 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:57:38.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.004+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe6d806ac30 msgr2=0x7fe6d806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.004+0000 7fe6f7200640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe6d806ac30 0x7fe6d806d0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8006fd0 tx=0x7fe6e8008040 comp rx=0 tx=0).stop 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6f81f5870 msgr2=0x7fe6f806daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6f81f5870 0x7fe6f806daf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40018a0 tx=0x7fe6e400dc00 comp rx=0 tx=0).stop 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 shutdown_connections 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6f81413d0 0x7fe6f8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6f81f5870 0x7fe6f806daf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe6d806ac30 0x7fe6d806d0f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 --2- 172.21.15.105:0/4153555715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6dc005180 0x7fe6f81f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 >> 172.21.15.105:0/4153555715 conn(0x7fe6f8065df0 msgr2=0x7fe6f813f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 shutdown_connections 2024-03-23T07:57:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.005+0000 7fe6f7200640 1 -- 172.21.15.105:0/4153555715 wait complete. 2024-03-23T07:57:38.024 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.225+0000 7f5e39c00640 1 Processor -- start 2024-03-23T07:57:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.225+0000 7f5e39c00640 1 -- start start 2024-03-23T07:57:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e39c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5e3406d390 0x7f5e3406d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e39c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e340b16a0 con 0x7f5e340adbc0 2024-03-23T07:57:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e39c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e340cbfb0 con 0x7f5e340cbab0 2024-03-23T07:57:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e39c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3406dd40 con 0x7f5e3406d390 2024-03-23T07:57:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e32a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5e340adbc0 0x7f5e340b41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55990/0 (socket says 172.21.15.105:55990) 2024-03-23T07:57:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e32a00640 1 -- 172.21.15.105:0/3124158486 learned_addr learned my addr 172.21.15.105:0/3124158486 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065963171 0 0) 0x7f5e340b16a0 con 0x7f5e340adbc0 2024-03-23T07:57:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e33400640 1 --2- 172.21.15.105:0/3124158486 >> v2:172.21.15.132:3300/0 conn(0x7f5e3406d390 0x7f5e3406d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.227+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e18005730 con 0x7f5e340adbc0 2024-03-23T07:57:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3853820680 0 0) 0x7f5e18005730 con 0x7f5e340adbc0 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 >> v2:172.21.15.132:3300/0 conn(0x7f5e3406d390 msgr2=0x7f5e3406d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 --2- 172.21.15.105:0/3124158486 >> v2:172.21.15.132:3300/0 conn(0x7f5e3406d390 0x7f5e3406d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 >> v1:172.21.15.132:6789/0 conn(0x7f5e340cbab0 legacy=0x7f5e340b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340cc290 con 0x7f5e340adbc0 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f5e24001670 con 0x7f5e340adbc0 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e33400640 1 --2- 172.21.15.105:0/3124158486 >> v2:172.21.15.132:3300/0 conn(0x7f5e3406d390 0x7f5e3406d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 >> v1:172.21.15.105:6789/0 conn(0x7f5e340adbc0 legacy=0x7f5e340b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e18005b80 0x7f5e18005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e180069d0 0x7f5e18006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e180074b0 0x7f5e1800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3406dd40 con 0x7f5e180069d0 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800e360 con 0x7f5e180074b0 2024-03-23T07:57:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800e520 con 0x7f5e18005b80 2024-03-23T07:57:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e2a000640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e180069d0 0x7f5e18006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e2a000640 1 -- 172.21.15.105:0/3124158486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e18005b80 msgr2=0x7f5e18005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e2a000640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e18005b80 0x7f5e18005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e2a000640 1 -- 172.21.15.105:0/3124158486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e180074b0 msgr2=0x7f5e1800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e2a000640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e180074b0 0x7f5e1800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e2a000640 1 -- 172.21.15.105:0/3124158486 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18005730 con 0x7f5e180069d0 2024-03-23T07:57:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.228+0000 7f5e33400640 1 -- 172.21.15.105:0/3124158486 reap_dead start 2024-03-23T07:57:38.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.229+0000 7f5e2a000640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e180069d0 0x7f5e18006e50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e340cbee0 tx=0x7f5e18007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb52823fbe83e85f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:38.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c032600 con 0x7f5e180069d0 2024-03-23T07:57:38.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c02e050 con 0x7f5e180069d0 2024-03-23T07:57:38.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e32000640 1 -- 172.21.15.105:0/3124158486 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0329e0 con 0x7f5e180069d0 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e39c00640 1 -- 172.21.15.105:0/3124158486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e180069d0 msgr2=0x7f5e18006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e39c00640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e180069d0 0x7f5e18006e50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e340cbee0 tx=0x7f5e18007a00 comp rx=0 tx=0).stop 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e39c00640 1 -- 172.21.15.105:0/3124158486 shutdown_connections 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e39c00640 1 --2- 172.21.15.105:0/3124158486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e180069d0 0x7f5e18006e50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e39c00640 1 -- 172.21.15.105:0/3124158486 >> 172.21.15.105:0/3124158486 conn(0x7f5e3406a3d0 msgr2=0x7f5e340b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e39c00640 1 -- 172.21.15.105:0/3124158486 shutdown_connections 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.230+0000 7f5e39c00640 1 -- 172.21.15.105:0/3124158486 wait complete. 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.232+0000 7f5e39c00640 1 Processor -- start 2024-03-23T07:57:38.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.232+0000 7f5e39c00640 1 -- start start 2024-03-23T07:57:38.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e39c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e180069d0 0x7f5e340acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e39c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e340aa720 0x7f5e34132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e39c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3406c280 0x7f5e3406c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e39c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340ad5a0 con 0x7f5e340aa720 2024-03-23T07:57:38.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e39c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3406cc20 con 0x7f5e3406c280 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e39c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340a8020 con 0x7f5e180069d0 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e2a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e340aa720 0x7f5e34132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e2a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e340aa720 0x7f5e34132bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52220/0 (socket says 172.21.15.105:52220) 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.233+0000 7f5e2a000640 1 -- 172.21.15.105:0/1913765614 learned_addr learned my addr 172.21.15.105:0/1913765614 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e33400640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3406c280 0x7f5e3406c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e2a000640 1 -- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e180069d0 msgr2=0x7f5e340acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e2a000640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e180069d0 0x7f5e340acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e2a000640 1 -- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3406c280 msgr2=0x7f5e3406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e2a000640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3406c280 0x7f5e3406c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e2a000640 1 -- 172.21.15.105:0/1913765614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340ad2c0 con 0x7f5e340aa720 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e33400640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3406c280 0x7f5e3406c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e2a000640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e340aa720 0x7f5e34132bf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c003e70 tx=0x7f5e1c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.234+0000 7f5e2b400640 1 -- 172.21.15.105:0/1913765614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c03c410 con 0x7f5e340aa720 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.235+0000 7f5e2b400640 1 -- 172.21.15.105:0/1913765614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c03c570 con 0x7f5e340aa720 2024-03-23T07:57:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.235+0000 7f5e2b400640 1 -- 172.21.15.105:0/1913765614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c03c8e0 con 0x7f5e340aa720 2024-03-23T07:57:38.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.235+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e340a79d0 con 0x7f5e340aa720 2024-03-23T07:57:38.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.235+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34133130 con 0x7f5e340aa720 2024-03-23T07:57:38.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.236+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e341332f0 con 0x7f5e340aa720 2024-03-23T07:57:38.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.237+0000 7f5e2b400640 1 -- 172.21.15.105:0/1913765614 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5e1c039050 con 0x7f5e340aa720 2024-03-23T07:57:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.238+0000 7f5e2b400640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5e1006f010 0x7f5e100714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.238+0000 7f5e2b400640 1 -- 172.21.15.105:0/1913765614 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5e1c0ab0b0 con 0x7f5e340aa720 2024-03-23T07:57:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.238+0000 7f5e32a00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5e1006f010 0x7f5e100714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.239+0000 7f5e32a00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5e1006f010 0x7f5e100714d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000f60 tx=0x7f5e240015c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:38.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.243+0000 7f5e2b400640 1 -- 172.21.15.105:0/1913765614 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5e1c075ea0 con 0x7f5e340aa720 2024-03-23T07:57:38.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:57:38.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:57:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.443+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e340a7830 con 0x7f5e340aa720 2024-03-23T07:57:38.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.447+0000 7f5e2b400640 1 -- 172.21.15.105:0/1913765614 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f5e340a7830 con 0x7f5e340aa720 2024-03-23T07:57:38.448 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:38.448 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:38.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5e1006f010 msgr2=0x7f5e100714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5e1006f010 0x7f5e100714d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000f60 tx=0x7f5e240015c0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e340aa720 msgr2=0x7f5e34132bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e340aa720 0x7f5e34132bf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c003e70 tx=0x7f5e1c0036f0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 shutdown_connections 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3406c280 0x7f5e3406c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e340aa720 0x7f5e34132bf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5e1006f010 0x7f5e100714d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 --2- 172.21.15.105:0/1913765614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e180069d0 0x7f5e340acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.449+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 >> 172.21.15.105:0/1913765614 conn(0x7f5e3406a3d0 msgr2=0x7f5e34142c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.450+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 shutdown_connections 2024-03-23T07:57:38.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.450+0000 7f5e39c00640 1 -- 172.21.15.105:0/1913765614 wait complete. 2024-03-23T07:57:38.470 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T07:57:38.470 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:57:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.706+0000 7fa22dc00640 1 Processor -- start 2024-03-23T07:57:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.706+0000 7fa22dc00640 1 -- start start 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa22dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280adbc0 0x7fa2280b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa22dc00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2280cbab0 0x7fa2280b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa22dc00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2280b16a0 con 0x7fa22806d390 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa22dc00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2280b4700 con 0x7fa2280cbab0 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa22dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280cbfb0 con 0x7fa2280adbc0 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa226000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2280cbab0 0x7fa2280b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa227400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa22806d390 0x7fa2280b2aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55996/0 (socket says 172.21.15.105:55996) 2024-03-23T07:57:38.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa226000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2280cbab0 0x7fa2280b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52230/0 (socket says 172.21.15.105:52230) 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.707+0000 7fa227400640 1 -- 172.21.15.105:0/3485576657 learned_addr learned my addr 172.21.15.105:0/3485576657 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.708+0000 7fa226a00640 1 --2- 172.21.15.105:0/3485576657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280adbc0 0x7fa2280b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.708+0000 7fa226000640 1 -- 172.21.15.105:0/3485576657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280adbc0 msgr2=0x7fa2280b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.708+0000 7fa226000640 1 --2- 172.21.15.105:0/3485576657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280adbc0 0x7fa2280b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.708+0000 7fa226000640 1 -- 172.21.15.105:0/3485576657 >> v1:172.21.15.105:6789/0 conn(0x7fa22806d390 legacy=0x7fa2280b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.708+0000 7fa225600640 1 -- 172.21.15.105:0/3485576657 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447348648 0 0) 0x7fa2280b16a0 con 0x7fa22806d390 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.708+0000 7fa226000640 1 -- 172.21.15.105:0/3485576657 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2280cc290 con 0x7fa2280cbab0 2024-03-23T07:57:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.708+0000 7fa226a00640 1 --2- 172.21.15.105:0/3485576657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280adbc0 0x7fa2280b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.709+0000 7fa226000640 1 --2- 172.21.15.105:0/3485576657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2280cbab0 0x7fa2280b20f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa21400ba40 tx=0x7fa21402dff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bdbb56d7b92431 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.709+0000 7fa225600640 1 -- 172.21.15.105:0/3485576657 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21402eda0 con 0x7fa2280cbab0 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.709+0000 7fa225600640 1 -- 172.21.15.105:0/3485576657 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa214036440 con 0x7fa2280cbab0 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.709+0000 7fa225600640 1 -- 172.21.15.105:0/3485576657 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140367d0 con 0x7fa2280cbab0 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 -- 172.21.15.105:0/3485576657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2280cbab0 msgr2=0x7fa2280b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 --2- 172.21.15.105:0/3485576657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2280cbab0 0x7fa2280b20f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa21400ba40 tx=0x7fa21402dff0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 -- 172.21.15.105:0/3485576657 shutdown_connections 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 --2- 172.21.15.105:0/3485576657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2280cbab0 0x7fa2280b20f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 --2- 172.21.15.105:0/3485576657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280adbc0 0x7fa2280b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 -- 172.21.15.105:0/3485576657 >> 172.21.15.105:0/3485576657 conn(0x7fa22806a3d0 msgr2=0x7fa2280b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:38.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 -- 172.21.15.105:0/3485576657 shutdown_connections 2024-03-23T07:57:38.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.710+0000 7fa22dc00640 1 -- 172.21.15.105:0/3485576657 wait complete. 2024-03-23T07:57:38.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.712+0000 7fa22dc00640 1 Processor -- start 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.712+0000 7fa22dc00640 1 -- start start 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa22dc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22806d390 0x7fa228130810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa22dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2280adbc0 0x7fa228138180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa22dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280cbab0 0x7fa2281386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa22dc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280b0c90 con 0x7fa22806d390 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa22dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815ae80 con 0x7fa2280adbc0 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa22dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280b06d0 con 0x7fa2280cbab0 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22806d390 0x7fa228130810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22806d390 0x7fa228130810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52232/0 (socket says 172.21.15.105:52232) 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 -- 172.21.15.105:0/1366472365 learned_addr learned my addr 172.21.15.105:0/1366472365 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 -- 172.21.15.105:0/1366472365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280cbab0 msgr2=0x7fa2281386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280cbab0 0x7fa2281386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 -- 172.21.15.105:0/1366472365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2280adbc0 msgr2=0x7fa228138180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2280adbc0 0x7fa228138180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.713+0000 7fa226a00640 1 -- 172.21.15.105:0/1366472365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21400b6a0 con 0x7fa22806d390 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.714+0000 7fa226a00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22806d390 0x7fa228130810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c7f0 tx=0x7fa21800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.714+0000 7fa21ea00640 1 -- 172.21.15.105:0/1366472365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218007a30 con 0x7fa22806d390 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.714+0000 7fa21ea00640 1 -- 172.21.15.105:0/1366472365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa218007b90 con 0x7fa22806d390 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.714+0000 7fa21ea00640 1 -- 172.21.15.105:0/1366472365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21800ce40 con 0x7fa22806d390 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.714+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2280b09b0 con 0x7fa22806d390 2024-03-23T07:57:38.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.714+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22815e1e0 con 0x7fa22806d390 2024-03-23T07:57:38.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.715+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22815a970 con 0x7fa22806d390 2024-03-23T07:57:38.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.716+0000 7fa21ea00640 1 -- 172.21.15.105:0/1366472365 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa2180113f0 con 0x7fa22806d390 2024-03-23T07:57:38.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.717+0000 7fa21ea00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa20806ac30 0x7fa20806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:38.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.717+0000 7fa21ea00640 1 -- 172.21.15.105:0/1366472365 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa2180897d0 con 0x7fa22806d390 2024-03-23T07:57:38.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.717+0000 7fa226000640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa20806ac30 0x7fa20806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:38.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.718+0000 7fa226000640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa20806ac30 0x7fa20806d0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa214038000 tx=0x7fa214008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:38.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.722+0000 7fa21ea00640 1 -- 172.21.15.105:0/1366472365 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa22815a970 con 0x7fa22806d390 2024-03-23T07:57:38.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.953+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa22815a970 con 0x7fa22806d390 2024-03-23T07:57:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.955+0000 7fa21ea00640 1 -- 172.21.15.105:0/1366472365 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7fa22815a970 con 0x7fa22806d390 2024-03-23T07:57:38.955 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:38.956 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:57:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:57:38.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.957+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa20806ac30 msgr2=0x7fa20806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.957+0000 7fa22dc00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa20806ac30 0x7fa20806d0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa214038000 tx=0x7fa214008040 comp rx=0 tx=0).stop 2024-03-23T07:57:38.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.957+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22806d390 msgr2=0x7fa228130810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:38.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.957+0000 7fa22dc00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22806d390 0x7fa228130810 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c7f0 tx=0x7fa21800cbf0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 shutdown_connections 2024-03-23T07:57:38.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa20806ac30 0x7fa20806d0f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2280cbab0 0x7fa2281386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2280adbc0 0x7fa228138180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 --2- 172.21.15.105:0/1366472365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa22806d390 0x7fa228130810 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:38.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 >> 172.21.15.105:0/1366472365 conn(0x7fa22806a3d0 msgr2=0x7fa228141f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:38.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 shutdown_connections 2024-03-23T07:57:38.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:38.958+0000 7fa22dc00640 1 -- 172.21.15.105:0/1366472365 wait complete. 2024-03-23T07:57:38.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:57:39.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.280+0000 7f25a8200640 1 Processor -- start 2024-03-23T07:57:39.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.280+0000 7f25a8200640 1 -- start start 2024-03-23T07:57:39.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.282+0000 7f25a8200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a00b5a30 0x7f25a00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.282+0000 7f25a8200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f25a00b4350 0x7f25a00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.282+0000 7f25a8200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a00d1830 con 0x7f25a015cda0 2024-03-23T07:57:39.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.282+0000 7f25a8200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00b4d20 con 0x7f25a00b4350 2024-03-23T07:57:39.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.282+0000 7f25a8200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00b1f20 con 0x7f25a00b5a30 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.282+0000 7f25a4e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a00b5a30 0x7f25a00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.282+0000 7f25a6200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f25a00b4350 0x7f25a00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.283+0000 7f25a6200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f25a00b4350 0x7f25a00b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:43642/0 (socket says 172.21.15.105:43642) 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.283+0000 7f25a6200640 1 -- 172.21.15.105:0/3878833451 learned_addr learned my addr 172.21.15.105:0/3878833451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.283+0000 7f25a6200640 1 -- 172.21.15.105:0/3878833451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a00b5a30 msgr2=0x7f25a00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.283+0000 7f25a6200640 1 --2- 172.21.15.105:0/3878833451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a00b5a30 0x7f25a00b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.283+0000 7f25a6200640 1 -- 172.21.15.105:0/3878833451 >> v1:172.21.15.132:6789/0 conn(0x7f25a015cda0 legacy=0x7f25a00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.283+0000 7f25a6200640 1 -- 172.21.15.105:0/3878833451 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00d0c20 con 0x7f25a00b4350 2024-03-23T07:57:39.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.284+0000 7f25a6200640 1 --2- 172.21.15.105:0/3878833451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b4350 0x7f25a00b47e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2598009e50 tx=0x7f259802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db7c2659f9988a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:39.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.285+0000 7f259fe00640 1 -- 172.21.15.105:0/3878833451 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259802cb00 con 0x7f25a00b4350 2024-03-23T07:57:39.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.285+0000 7f259fe00640 1 -- 172.21.15.105:0/3878833451 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259802cc60 con 0x7f25a00b4350 2024-03-23T07:57:39.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.285+0000 7f259fe00640 1 -- 172.21.15.105:0/3878833451 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980343f0 con 0x7f25a00b4350 2024-03-23T07:57:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.285+0000 7f25a8200640 1 -- 172.21.15.105:0/3878833451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b4350 msgr2=0x7f25a00b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.285+0000 7f25a8200640 1 --2- 172.21.15.105:0/3878833451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b4350 0x7f25a00b47e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2598009e50 tx=0x7f259802bf60 comp rx=0 tx=0).stop 2024-03-23T07:57:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.286+0000 7f25a8200640 1 -- 172.21.15.105:0/3878833451 shutdown_connections 2024-03-23T07:57:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.286+0000 7f25a8200640 1 --2- 172.21.15.105:0/3878833451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b4350 0x7f25a00b47e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.286+0000 7f25a8200640 1 --2- 172.21.15.105:0/3878833451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a00b5a30 0x7f25a00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.286+0000 7f25a8200640 1 -- 172.21.15.105:0/3878833451 >> 172.21.15.105:0/3878833451 conn(0x7f25a00cf350 msgr2=0x7f25a00cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.286+0000 7f25a8200640 1 -- 172.21.15.105:0/3878833451 shutdown_connections 2024-03-23T07:57:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.286+0000 7f25a8200640 1 -- 172.21.15.105:0/3878833451 wait complete. 2024-03-23T07:57:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.287+0000 7f25a8200640 1 Processor -- start 2024-03-23T07:57:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.288+0000 7f25a8200640 1 -- start start 2024-03-23T07:57:39.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.288+0000 7f25a8200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25a00b4350 0x7f25a015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a8200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b5a30 0x7f25a0150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a8200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a015cda0 0x7f25a015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a8200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01547c0 con 0x7f25a00b4350 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a8200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a015d870 con 0x7f25a00b5a30 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a8200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a015e1b0 con 0x7f25a015cda0 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25a00b4350 0x7f25a015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25a00b4350 0x7f25a015db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52240/0 (socket says 172.21.15.105:52240) 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 -- 172.21.15.105:0/4025623937 learned_addr learned my addr 172.21.15.105:0/4025623937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a6200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a015cda0 0x7f25a015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a4e00640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b5a30 0x7f25a0150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 -- 172.21.15.105:0/4025623937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a015cda0 msgr2=0x7f25a015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a015cda0 0x7f25a015d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 -- 172.21.15.105:0/4025623937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b5a30 msgr2=0x7f25a0150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b5a30 0x7f25a0150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.289+0000 7f25a5800640 1 -- 172.21.15.105:0/4025623937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598009b30 con 0x7f25a00b4350 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.290+0000 7f25a5800640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25a00b4350 0x7f25a015db20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f259000c400 tx=0x7f259000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.290+0000 7f259d600640 1 -- 172.21.15.105:0/4025623937 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900052d0 con 0x7f25a00b4350 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.290+0000 7f259d600640 1 -- 172.21.15.105:0/4025623937 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590005430 con 0x7f25a00b4350 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.290+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a0153e90 con 0x7f25a00b4350 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.290+0000 7f259d600640 1 -- 172.21.15.105:0/4025623937 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590003830 con 0x7f25a00b4350 2024-03-23T07:57:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.290+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a0154450 con 0x7f25a00b4350 2024-03-23T07:57:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.291+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a0154610 con 0x7f25a00b4350 2024-03-23T07:57:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.292+0000 7f259d600640 1 -- 172.21.15.105:0/4025623937 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2590003990 con 0x7f25a00b4350 2024-03-23T07:57:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.293+0000 7f259d600640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f258006ac00 0x7f258006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.293+0000 7f259d600640 1 -- 172.21.15.105:0/4025623937 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2590088c10 con 0x7f25a00b4350 2024-03-23T07:57:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.293+0000 7f25a4e00640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f258006ac00 0x7f258006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.294+0000 7f25a4e00640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f258006ac00 0x7f258006d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25a01563b0 tx=0x7f258c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:39.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.297+0000 7f259d600640 1 -- 172.21.15.105:0/4025623937 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2590053ab0 con 0x7f25a00b4350 2024-03-23T07:57:39.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.530+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25a0073390 con 0x7f25a00b4350 2024-03-23T07:57:39.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.531+0000 7f259d600640 1 -- 172.21.15.105:0/4025623937 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4739 (secure 0 0 0) 0x7f2590017090 con 0x7f25a00b4350 2024-03-23T07:57:39.532 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:39.532 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2300828775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2300828775},{"type":"v1","addr":"172.21.15.132:6835","nonce":2300828775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2862635354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2862635354},{"type":"v1","addr":"172.21.15.105:6837","nonce":2862635354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3961678308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3961678308},{"type":"v1","addr":"172.21.15.105:6836","nonce":3961678308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:56:17.694756+0000","modified":"2024-03-23T07:56:45.123427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/713904573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":713904573},{"type":"v1","addr":"172.21.15.132:6834","nonce":713904573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T07:57:39.532 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-23T07:57:39.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.533+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f258006ac00 msgr2=0x7f258006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.533+0000 7f25a8200640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f258006ac00 0x7f258006d0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25a01563b0 tx=0x7f258c009190 comp rx=0 tx=0).stop 2024-03-23T07:57:39.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25a00b4350 msgr2=0x7f25a015db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25a00b4350 0x7f25a015db20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f259000c400 tx=0x7f259000c8c0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 shutdown_connections 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f258006ac00 0x7f258006d0c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a015cda0 0x7f25a015d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a00b5a30 0x7f25a0150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 --2- 172.21.15.105:0/4025623937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25a00b4350 0x7f25a015db20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 >> 172.21.15.105:0/4025623937 conn(0x7f25a00cf350 msgr2=0x7f25a00ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 shutdown_connections 2024-03-23T07:57:39.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.534+0000 7f25a8200640 1 -- 172.21.15.105:0/4025623937 wait complete. 2024-03-23T07:57:39.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.758+0000 7f3a0a800640 1 Processor -- start 2024-03-23T07:57:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.758+0000 7f3a0a800640 1 -- start start 2024-03-23T07:57:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.759+0000 7f3a0a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 0x7f3a0415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.759+0000 7f3a0a800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0415f320 con 0x7f3a040b39b0 2024-03-23T07:57:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.760+0000 7f3a0a800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0406c6f0 con 0x7f3a0415d410 2024-03-23T07:57:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.760+0000 7f3a0a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0406c9d0 con 0x7f3a0415b560 2024-03-23T07:57:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.760+0000 7f3a02a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3a040b39b0 0x7f3a0415e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56010/0 (socket says 172.21.15.105:56010) 2024-03-23T07:57:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.760+0000 7f3a02a00640 1 -- 172.21.15.105:0/624209380 learned_addr learned my addr 172.21.15.105:0/624209380 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.760+0000 7f3a03400640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 0x7f3a0415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.760+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285051492 0 0) 0x7f3a0415f320 con 0x7f3a040b39b0 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.760+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e8003520 con 0x7f3a040b39b0 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2496143626 0 0) 0x7f39e8003520 con 0x7f3a040b39b0 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 msgr2=0x7f3a0415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 0x7f3a0415b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 >> v1:172.21.15.132:6789/0 conn(0x7f3a0415d410 legacy=0x7f3a0415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0406ccb0 con 0x7f3a040b39b0 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f39f0001670 con 0x7f3a040b39b0 2024-03-23T07:57:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 >> v1:172.21.15.105:6789/0 conn(0x7f3a040b39b0 legacy=0x7f3a0415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39e80039a0 0x7f39e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 0x7f39e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39e8004d00 0x7f39e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0406c9d0 con 0x7f39e8004d00 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e800ba30 con 0x7f39e80039a0 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e800bbf0 con 0x7f3a0415b560 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a03e00640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39e8004d00 0x7f39e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a02a00640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 0x7f39e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.761+0000 7f3a03400640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39e80039a0 0x7f39e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.762+0000 7f3a03e00640 1 -- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 msgr2=0x7f39e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.762+0000 7f3a03e00640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0415b560 0x7f39e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.762+0000 7f3a03e00640 1 -- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39e80039a0 msgr2=0x7f39e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.762+0000 7f3a03e00640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39e80039a0 0x7f39e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.762+0000 7f3a03e00640 1 -- 172.21.15.105:0/624209380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8003520 con 0x7f39e8004d00 2024-03-23T07:57:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.762+0000 7f3a03400640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39e80039a0 0x7f39e8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.763+0000 7f3a03e00640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39e8004d00 0x7f39e8005180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415d8a0 tx=0x7f39e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde65eed072e2b9c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.763+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc003b70 con 0x7f39e8004d00 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.763+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc003cd0 con 0x7f39e8004d00 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.763+0000 7f3a02000640 1 -- 172.21.15.105:0/624209380 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc005310 con 0x7f39e8004d00 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.764+0000 7f3a0a800640 1 -- 172.21.15.105:0/624209380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39e8004d00 msgr2=0x7f39e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.764+0000 7f3a0a800640 1 --2- 172.21.15.105:0/624209380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39e8004d00 0x7f39e8005180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415d8a0 tx=0x7f39e80057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.764+0000 7f3a03e00640 1 -- 172.21.15.105:0/624209380 reap_dead start 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.764+0000 7f3a0a800640 1 -- 172.21.15.105:0/624209380 shutdown_connections 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.764+0000 7f3a0a800640 1 -- 172.21.15.105:0/624209380 >> 172.21.15.105:0/624209380 conn(0x7f3a0406a3d0 msgr2=0x7f3a040b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.764+0000 7f3a0a800640 1 -- 172.21.15.105:0/624209380 shutdown_connections 2024-03-23T07:57:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.764+0000 7f3a0a800640 1 -- 172.21.15.105:0/624209380 wait complete. 2024-03-23T07:57:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.766+0000 7f3a0a800640 1 Processor -- start 2024-03-23T07:57:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.766+0000 7f3a0a800640 1 -- start start 2024-03-23T07:57:39.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a0a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a04159ba0 0x7f3a040cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a0a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a04159f80 0x7f3a041f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a0a800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a04154ac0 0x7f3a041f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a0a800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040af370 con 0x7f3a04159ba0 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a0a800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a04154630 con 0x7f3a04154ac0 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a0a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040b2330 con 0x7f3a04159f80 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a04159ba0 0x7f3a040cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a04159ba0 0x7f3a040cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52254/0 (socket says 172.21.15.105:52254) 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 -- 172.21.15.105:0/4219606203 learned_addr learned my addr 172.21.15.105:0/4219606203 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a02a00640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a04159f80 0x7f3a041f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03e00640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a04154ac0 0x7f3a041f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 -- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a04159f80 msgr2=0x7f3a041f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a04159f80 0x7f3a041f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 -- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a04154ac0 msgr2=0x7f3a041f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a04154ac0 0x7f3a041f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.768+0000 7f3a03400640 1 -- 172.21.15.105:0/4219606203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a040b2610 con 0x7f3a04159ba0 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.769+0000 7f3a03400640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a04159ba0 0x7f3a040cd700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f39f4011630 tx=0x7f39f4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.769+0000 7f39fb400640 1 -- 172.21.15.105:0/4219606203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f400f920 con 0x7f3a04159ba0 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.769+0000 7f39fb400640 1 -- 172.21.15.105:0/4219606203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f400fa80 con 0x7f3a04159ba0 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.769+0000 7f39fb400640 1 -- 172.21.15.105:0/4219606203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f400fe10 con 0x7f3a04159ba0 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.769+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a041542c0 con 0x7f3a04159ba0 2024-03-23T07:57:39.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.769+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a04140130 con 0x7f3a04159ba0 2024-03-23T07:57:39.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.770+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a04154450 con 0x7f3a04159ba0 2024-03-23T07:57:39.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.771+0000 7f39fb400640 1 -- 172.21.15.105:0/4219606203 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f39f4015460 con 0x7f3a04159ba0 2024-03-23T07:57:39.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.772+0000 7f39fb400640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f39e406ac30 0x7f39e406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:39.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.773+0000 7f39fb400640 1 -- 172.21.15.105:0/4219606203 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f39f408a080 con 0x7f3a04159ba0 2024-03-23T07:57:39.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.773+0000 7f3a02a00640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f39e406ac30 0x7f39e406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:39.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.773+0000 7f3a02a00640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f39e406ac30 0x7f39e406d0f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415b000 tx=0x7f39f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:39.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.777+0000 7f39fb400640 1 -- 172.21.15.105:0/4219606203 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f39f40bd580 con 0x7f3a04159ba0 2024-03-23T07:57:39.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.970+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a0406ac20 con 0x7f3a04159ba0 2024-03-23T07:57:39.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.973+0000 7f39fb400640 1 -- 172.21.15.105:0/4219606203 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18071 (secure 0 0 0) 0x7f39f4022020 con 0x7f3a04159ba0 2024-03-23T07:57:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:39.974 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:56:59.965833+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:56:15.951945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T07:56:16.662976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-23T07:56:20.014100+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.b","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f39e406ac30 msgr2=0x7f39e406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f39e406ac30 0x7f39e406d0f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415b000 tx=0x7f39f0009040 comp rx=0 tx=0).stop 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a04159ba0 msgr2=0x7f3a040cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a04159ba0 0x7f3a040cd700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f39f4011630 tx=0x7f39f4011af0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 shutdown_connections 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a04154ac0 0x7f3a041f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f39e406ac30 0x7f39e406d0f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a04159f80 0x7f3a041f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 --2- 172.21.15.105:0/4219606203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a04159ba0 0x7f3a040cd700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:39.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 >> 172.21.15.105:0/4219606203 conn(0x7f3a0406a3d0 msgr2=0x7f3a0415e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:39.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.975+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 shutdown_connections 2024-03-23T07:57:39.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:39.976+0000 7f3a0a800640 1 -- 172.21.15.105:0/4219606203 wait complete. 2024-03-23T07:57:39.995 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T07:57:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.206+0000 7f693ea00640 1 Processor -- start 2024-03-23T07:57:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.206+0000 7f693ea00640 1 -- start start 2024-03-23T07:57:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.208+0000 7f693ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380ce930 0x7f69380ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.208+0000 7f693ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f693806d2c0 0x7f69380cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.208+0000 7f693ea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69380cf250 con 0x7f69380cb660 2024-03-23T07:57:40.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.208+0000 7f693ea00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693815bae0 con 0x7f693806d2c0 2024-03-23T07:57:40.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.208+0000 7f693ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693815ae60 con 0x7f69380ce930 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f6937e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380ce930 0x7f69380ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f693ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f693806d2c0 0x7f69380cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f693ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f693806d2c0 0x7f69380cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:43678/0 (socket says 172.21.15.105:43678) 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f693ca00640 1 -- 172.21.15.105:0/2006631999 learned_addr learned my addr 172.21.15.105:0/2006631999 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f693ca00640 1 -- 172.21.15.105:0/2006631999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380ce930 msgr2=0x7f69380ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f693ca00640 1 --2- 172.21.15.105:0/2006631999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380ce930 0x7f69380ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f693ca00640 1 -- 172.21.15.105:0/2006631999 >> v1:172.21.15.132:6789/0 conn(0x7f69380cb660 legacy=0x7f6938070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.209+0000 7f693ca00640 1 -- 172.21.15.105:0/2006631999 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693815b140 con 0x7f693806d2c0 2024-03-23T07:57:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.211+0000 7f693ca00640 1 --2- 172.21.15.105:0/2006631999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693806d2c0 0x7f69380cbae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6930009e50 tx=0x7f693002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c85dc80ee87edf5a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.211+0000 7f6936a00640 1 -- 172.21.15.105:0/2006631999 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693002caf0 con 0x7f693806d2c0 2024-03-23T07:57:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.211+0000 7f6936a00640 1 -- 172.21.15.105:0/2006631999 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693002cc50 con 0x7f693806d2c0 2024-03-23T07:57:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.211+0000 7f6936a00640 1 -- 172.21.15.105:0/2006631999 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69300343f0 con 0x7f693806d2c0 2024-03-23T07:57:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 -- 172.21.15.105:0/2006631999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693806d2c0 msgr2=0x7f69380cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 --2- 172.21.15.105:0/2006631999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693806d2c0 0x7f69380cbae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6930009e50 tx=0x7f693002bf50 comp rx=0 tx=0).stop 2024-03-23T07:57:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 -- 172.21.15.105:0/2006631999 shutdown_connections 2024-03-23T07:57:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 --2- 172.21.15.105:0/2006631999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f693806d2c0 0x7f69380cbae0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 --2- 172.21.15.105:0/2006631999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380ce930 0x7f69380ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 -- 172.21.15.105:0/2006631999 >> 172.21.15.105:0/2006631999 conn(0x7f693806a0b0 msgr2=0x7f693806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 -- 172.21.15.105:0/2006631999 shutdown_connections 2024-03-23T07:57:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.212+0000 7f693ea00640 1 -- 172.21.15.105:0/2006631999 wait complete. 2024-03-23T07:57:40.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.214+0000 7f693ea00640 1 Processor -- start 2024-03-23T07:57:40.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.214+0000 7f693ea00640 1 -- start start 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f693ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693806d2c0 0x7f6938157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f693ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380cb660 0x7f693814e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f693ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69380ce930 0x7f6938150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f693ea00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6938157ab0 con 0x7f693806d2c0 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f693ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6938155590 con 0x7f69380ce930 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f693ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6938155870 con 0x7f69380cb660 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f6937e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693806d2c0 0x7f6938157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f6937e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693806d2c0 0x7f6938157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52258/0 (socket says 172.21.15.105:52258) 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f6937e00640 1 -- 172.21.15.105:0/3859412001 learned_addr learned my addr 172.21.15.105:0/3859412001 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f693ca00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69380ce930 0x7f6938150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.215+0000 7f6937400640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380cb660 0x7f693814e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f6937e00640 1 -- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380cb660 msgr2=0x7f693814e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f6937e00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380cb660 0x7f693814e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f6937e00640 1 -- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69380ce930 msgr2=0x7f6938150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f6937e00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69380ce930 0x7f6938150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f6937e00640 1 -- 172.21.15.105:0/3859412001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6930009b30 con 0x7f693806d2c0 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f6937400640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380cb660 0x7f693814e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f693ca00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69380ce930 0x7f6938150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f6937e00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693806d2c0 0x7f6938157240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f692800e9b0 tx=0x7f692800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f692fe00640 1 -- 172.21.15.105:0/3859412001 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928009670 con 0x7f693806d2c0 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f692fe00640 1 -- 172.21.15.105:0/3859412001 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692800c9a0 con 0x7f693806d2c0 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f692fe00640 1 -- 172.21.15.105:0/3859412001 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692800cd50 con 0x7f693806d2c0 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6938155280 con 0x7f693806d2c0 2024-03-23T07:57:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.216+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6938155e30 con 0x7f693806d2c0 2024-03-23T07:57:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.218+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6938073390 con 0x7f693806d2c0 2024-03-23T07:57:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.218+0000 7f692fe00640 1 -- 172.21.15.105:0/3859412001 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6928013470 con 0x7f693806d2c0 2024-03-23T07:57:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.219+0000 7f692fe00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691806ac00 0x7f691806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.219+0000 7f692fe00640 1 -- 172.21.15.105:0/3859412001 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6928089a60 con 0x7f693806d2c0 2024-03-23T07:57:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.220+0000 7f6937400640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691806ac00 0x7f691806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.220+0000 7f6937400640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691806ac00 0x7f691806d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69381583d0 tx=0x7f6924009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.224+0000 7f692fe00640 1 -- 172.21.15.105:0/3859412001 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f69280547d0 con 0x7f693806d2c0 2024-03-23T07:57:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.457+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f693806da20 con 0x7f693806d2c0 2024-03-23T07:57:40.459 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-03-23T07:57:40.459+0000 7f31aa200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T07:57:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.476+0000 7f692fe00640 1 -- 172.21.15.105:0/3859412001 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6928057df0 con 0x7f693806d2c0 2024-03-23T07:57:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T07:57:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.478+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691806ac00 msgr2=0x7f691806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.478+0000 7f693ea00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691806ac00 0x7f691806d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69381583d0 tx=0x7f6924009040 comp rx=0 tx=0).stop 2024-03-23T07:57:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.478+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693806d2c0 msgr2=0x7f6938157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693806d2c0 0x7f6938157240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f692800e9b0 tx=0x7f692800ee70 comp rx=0 tx=0).stop 2024-03-23T07:57:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 shutdown_connections 2024-03-23T07:57:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69380ce930 0x7f6938150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f691806ac00 0x7f691806d0c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69380cb660 0x7f693814e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 --2- 172.21.15.105:0/3859412001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f693806d2c0 0x7f6938157240 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 >> 172.21.15.105:0/3859412001 conn(0x7f693806a0b0 msgr2=0x7f69380ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 shutdown_connections 2024-03-23T07:57:40.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.479+0000 7f693ea00640 1 -- 172.21.15.105:0/3859412001 wait complete. 2024-03-23T07:57:40.501 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T07:57:40.506 INFO:tasks.ceph.mds.b.smithi132.stderr: -5697> 2024-03-23T07:56:19.044+0000 7f4598c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T07:57:40.589 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-03-23T07:57:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.697+0000 7f57e4a00640 1 Processor -- start 2024-03-23T07:57:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.697+0000 7f57e4a00640 1 -- start start 2024-03-23T07:57:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.698+0000 7f57e4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 0x7f57e015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.698+0000 7f57e4a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00c8c90 con 0x7f57e0159cd0 2024-03-23T07:57:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.698+0000 7f57e4a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00cc6c0 con 0x7f57e00cc200 2024-03-23T07:57:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.698+0000 7f57e4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00cc9a0 con 0x7f57e00b26c0 2024-03-23T07:57:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.699+0000 7f57de800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f57e0159cd0 0x7f57e00ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56012/0 (socket says 172.21.15.105:56012) 2024-03-23T07:57:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.699+0000 7f57de800640 1 -- 172.21.15.105:0/4190756280 learned_addr learned my addr 172.21.15.105:0/4190756280 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.699+0000 7f57df200640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 0x7f57e015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.699+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516385541 0 0) 0x7f57e00c8c90 con 0x7f57e0159cd0 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.699+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c4003520 con 0x7f57e0159cd0 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2599984748 0 0) 0x7f57c4003520 con 0x7f57e0159cd0 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 msgr2=0x7f57e015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 0x7f57e015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 >> v1:172.21.15.132:6789/0 conn(0x7f57e00cc200 legacy=0x7f57e00cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00ccc80 con 0x7f57e0159cd0 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f57d0001670 con 0x7f57e0159cd0 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57df200640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 0x7f57e015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 >> v1:172.21.15.105:6789/0 conn(0x7f57e0159cd0 legacy=0x7f57e00ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 0x7f57c40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4003f90 0x7f57c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4004d00 0x7f57c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00cc9a0 con 0x7f57c4003f90 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c400bae0 con 0x7f57c4004d00 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c400bca0 con 0x7f57e00b26c0 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57dde00640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4003f90 0x7f57c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.700+0000 7f57de800640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 0x7f57c40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.701+0000 7f57df200640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4004d00 0x7f57c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.701+0000 7f57dde00640 1 -- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 msgr2=0x7f57c40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.701+0000 7f57dde00640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00b26c0 0x7f57c40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.701+0000 7f57dde00640 1 -- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4004d00 msgr2=0x7f57c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.701+0000 7f57dde00640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4004d00 0x7f57c4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.701+0000 7f57dde00640 1 -- 172.21.15.105:0/4190756280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4003520 con 0x7f57c4003f90 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57dde00640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4003f90 0x7f57c40047c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f57cc001750 tx=0x7f57c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3def40c3554d85d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0052d0 con 0x7f57c4003f90 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc02e050 con 0x7f57c4003f90 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57dd400640 1 -- 172.21.15.105:0/4190756280 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc003830 con 0x7f57c4003f90 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57e4a00640 1 -- 172.21.15.105:0/4190756280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4003f90 msgr2=0x7f57c40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57e4a00640 1 --2- 172.21.15.105:0/4190756280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4003f90 0x7f57c40047c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f57cc001750 tx=0x7f57c40057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57df200640 1 -- 172.21.15.105:0/4190756280 reap_dead start 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57e4a00640 1 -- 172.21.15.105:0/4190756280 shutdown_connections 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.702+0000 7f57e4a00640 1 -- 172.21.15.105:0/4190756280 >> 172.21.15.105:0/4190756280 conn(0x7f57e00b01b0 msgr2=0x7f57e00b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.703+0000 7f57e4a00640 1 -- 172.21.15.105:0/4190756280 shutdown_connections 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.703+0000 7f57e4a00640 1 -- 172.21.15.105:0/4190756280 wait complete. 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.704+0000 7f57e4a00640 1 Processor -- start 2024-03-23T07:57:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.704+0000 7f57e4a00640 1 -- start start 2024-03-23T07:57:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57e4a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4003f90 0x7f57e009bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57e4a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e015b9f0 0x7f57e0099e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57e4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00ae0f0 0x7f57e0152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57e4a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0098960 con 0x7f57e015b9f0 2024-03-23T07:57:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57e4a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e009c3e0 con 0x7f57c4003f90 2024-03-23T07:57:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57e4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00983c0 con 0x7f57e00ae0f0 2024-03-23T07:57:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57dde00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e015b9f0 0x7f57e0099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57dde00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e015b9f0 0x7f57e0099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52282/0 (socket says 172.21.15.105:52282) 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.705+0000 7f57dde00640 1 -- 172.21.15.105:0/360428500 learned_addr learned my addr 172.21.15.105:0/360428500 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57df200640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00ae0f0 0x7f57e0152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57de800640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4003f90 0x7f57e009bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57dde00640 1 -- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00ae0f0 msgr2=0x7f57e0152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57dde00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00ae0f0 0x7f57e0152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57dde00640 1 -- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4003f90 msgr2=0x7f57e009bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57dde00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4003f90 0x7f57e009bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57dde00640 1 -- 172.21.15.105:0/360428500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0098680 con 0x7f57e015b9f0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57dde00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e015b9f0 0x7f57e0099e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f57cc008fd0 tx=0x7f57cc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57d6a00640 1 -- 172.21.15.105:0/360428500 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc007a30 con 0x7f57e015b9f0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57d6a00640 1 -- 172.21.15.105:0/360428500 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0069e0 con 0x7f57e015b9f0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e006cbf0 con 0x7f57e015b9f0 2024-03-23T07:57:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.706+0000 7f57d6a00640 1 -- 172.21.15.105:0/360428500 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc006d50 con 0x7f57e015b9f0 2024-03-23T07:57:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.707+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e006d1b0 con 0x7f57e015b9f0 2024-03-23T07:57:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.708+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e006ca50 con 0x7f57e015b9f0 2024-03-23T07:57:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.709+0000 7f57d6a00640 1 -- 172.21.15.105:0/360428500 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f57cc0035f0 con 0x7f57e015b9f0 2024-03-23T07:57:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.709+0000 7f57d6a00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f57c006ac30 0x7f57c006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.710+0000 7f57d6a00640 1 -- 172.21.15.105:0/360428500 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6633+0+0 (secure 0 0 0) 0x7f57cc0ab8c0 con 0x7f57e015b9f0 2024-03-23T07:57:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.710+0000 7f57de800640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f57c006ac30 0x7f57c006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.710+0000 7f57de800640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f57c006ac30 0x7f57c006d0f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57d0001050 tx=0x7f57d0001d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:40.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.714+0000 7f57d6a00640 1 -- 172.21.15.105:0/360428500 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f57e006ca50 con 0x7f57e015b9f0 2024-03-23T07:57:40.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:40.946+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57e006ca50 con 0x7f57e015b9f0 2024-03-23T07:57:41.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.479+0000 7f57d6a00640 1 -- 172.21.15.105:0/360428500 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57e006ca50 con 0x7f57e015b9f0 2024-03-23T07:57:41.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f57c006ac30 msgr2=0x7f57c006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f57c006ac30 0x7f57c006d0f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57d0001050 tx=0x7f57d0001d30 comp rx=0 tx=0).stop 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e015b9f0 msgr2=0x7f57e0099e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e015b9f0 0x7f57e0099e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f57cc008fd0 tx=0x7f57cc001120 comp rx=0 tx=0).stop 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 shutdown_connections 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f57c006ac30 0x7f57c006d0f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e00ae0f0 0x7f57e0152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e015b9f0 0x7f57e0099e00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.483+0000 7f57e4a00640 1 --2- 172.21.15.105:0/360428500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c4003f90 0x7f57e009bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.484+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 >> 172.21.15.105:0/360428500 conn(0x7f57e00b01b0 msgr2=0x7f57e00ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.484+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 shutdown_connections 2024-03-23T07:57:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.484+0000 7f57e4a00640 1 -- 172.21.15.105:0/360428500 wait complete. 2024-03-23T07:57:41.505 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T07:57:41.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.710+0000 7fbb8f000640 1 Processor -- start 2024-03-23T07:57:41.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.710+0000 7fbb8f000640 1 -- start start 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb8f000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb880d2400 0x7fbb880cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb8f000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8815cfd0 con 0x7fbb8815cbf0 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb8f000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb880d1100 con 0x7fbb880b8670 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb8f000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb880d1360 con 0x7fbb880d2400 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb87400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb880d2400 0x7fbb880cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb87e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbb8815cbf0 0x7fbb880d09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56028/0 (socket says 172.21.15.105:56028) 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb87400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb880d2400 0x7fbb880cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52290/0 (socket says 172.21.15.105:52290) 2024-03-23T07:57:41.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.712+0000 7fbb87e00640 1 -- 172.21.15.105:0/763478557 learned_addr learned my addr 172.21.15.105:0/763478557 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.713+0000 7fbb87400640 1 -- 172.21.15.105:0/763478557 >> v1:172.21.15.132:6789/0 conn(0x7fbb880b8670 legacy=0x7fbb880b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.713+0000 7fbb87400640 1 -- 172.21.15.105:0/763478557 >> v1:172.21.15.105:6789/0 conn(0x7fbb8815cbf0 legacy=0x7fbb880d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.713+0000 7fbb87400640 1 -- 172.21.15.105:0/763478557 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb880d1640 con 0x7fbb880d2400 2024-03-23T07:57:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.713+0000 7fbb86a00640 1 -- 172.21.15.105:0/763478557 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926960888 0 0) 0x7fbb78001110 con 0x7fbb8815cbf0 2024-03-23T07:57:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.714+0000 7fbb87400640 1 --2- 172.21.15.105:0/763478557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880d2400 0x7fbb880cd6d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097a0 tx=0x7fbb7002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8fd4482535dd07 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.714+0000 7fbb86a00640 1 -- 172.21.15.105:0/763478557 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70030610 con 0x7fbb880d2400 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.714+0000 7fbb86a00640 1 -- 172.21.15.105:0/763478557 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb70030770 con 0x7fbb880d2400 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.714+0000 7fbb86a00640 1 -- 172.21.15.105:0/763478557 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70030b00 con 0x7fbb880d2400 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.715+0000 7fbb8f000640 1 -- 172.21.15.105:0/763478557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880d2400 msgr2=0x7fbb880cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.715+0000 7fbb8f000640 1 --2- 172.21.15.105:0/763478557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880d2400 0x7fbb880cd6d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097a0 tx=0x7fbb7002b960 comp rx=0 tx=0).stop 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.715+0000 7fbb8f000640 1 -- 172.21.15.105:0/763478557 shutdown_connections 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.715+0000 7fbb8f000640 1 --2- 172.21.15.105:0/763478557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880d2400 0x7fbb880cd6d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.715+0000 7fbb8f000640 1 -- 172.21.15.105:0/763478557 >> 172.21.15.105:0/763478557 conn(0x7fbb880cf560 msgr2=0x7fbb880cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.715+0000 7fbb8f000640 1 -- 172.21.15.105:0/763478557 shutdown_connections 2024-03-23T07:57:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.715+0000 7fbb8f000640 1 -- 172.21.15.105:0/763478557 wait complete. 2024-03-23T07:57:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.717+0000 7fbb8f000640 1 Processor -- start 2024-03-23T07:57:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.717+0000 7fbb8f000640 1 -- start start 2024-03-23T07:57:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.718+0000 7fbb8f000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880b8670 0x7fbb880c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.718+0000 7fbb8f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb880d2400 0x7fbb881f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.718+0000 7fbb8f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb8815cbf0 0x7fbb8815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.718+0000 7fbb8f000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881f2f10 con 0x7fbb880b8670 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.718+0000 7fbb8f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8815eb70 con 0x7fbb880d2400 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.718+0000 7fbb8f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8815ee50 con 0x7fbb8815cbf0 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.718+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880b8670 0x7fbb880c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880b8670 0x7fbb880c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52296/0 (socket says 172.21.15.105:52296) 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 -- 172.21.15.105:0/3687023724 learned_addr learned my addr 172.21.15.105:0/3687023724 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87400640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb880d2400 0x7fbb881f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb8d000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb8815cbf0 0x7fbb8815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 -- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb8815cbf0 msgr2=0x7fbb8815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb8815cbf0 0x7fbb8815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 -- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb880d2400 msgr2=0x7fbb881f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb880d2400 0x7fbb881f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 -- 172.21.15.105:0/3687023724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70009400 con 0x7fbb880b8670 2024-03-23T07:57:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb8d000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb8815cbf0 0x7fbb8815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb87e00640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880b8670 0x7fbb880c8e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c430 tx=0x7fbb7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb7fe00640 1 -- 172.21.15.105:0/3687023724 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb780074e0 con 0x7fbb880b8670 2024-03-23T07:57:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb7fe00640 1 -- 172.21.15.105:0/3687023724 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb78007660 con 0x7fbb880b8670 2024-03-23T07:57:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.719+0000 7fbb7fe00640 1 -- 172.21.15.105:0/3687023724 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78004a20 con 0x7fbb880b8670 2024-03-23T07:57:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.720+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8815f130 con 0x7fbb880b8670 2024-03-23T07:57:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.720+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8815f6f0 con 0x7fbb880b8670 2024-03-23T07:57:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.721+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb88073390 con 0x7fbb880b8670 2024-03-23T07:57:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.722+0000 7fbb7fe00640 1 -- 172.21.15.105:0/3687023724 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbb78004b80 con 0x7fbb880b8670 2024-03-23T07:57:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.722+0000 7fbb7fe00640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb6406ac00 0x7fbb6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.723+0000 7fbb7fe00640 1 -- 172.21.15.105:0/3687023724 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6633+0+0 (secure 0 0 0) 0x7fbb78089430 con 0x7fbb880b8670 2024-03-23T07:57:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.723+0000 7fbb87400640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb6406ac00 0x7fbb6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.723+0000 7fbb87400640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb6406ac00 0x7fbb6406d0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb70036000 tx=0x7fbb70008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.727+0000 7fbb7fe00640 1 -- 172.21.15.105:0/3687023724 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbb780541d0 con 0x7fbb880b8670 2024-03-23T07:57:41.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:41.922+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb8815cfd0 con 0x7fbb880b8670 2024-03-23T07:57:42.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.483+0000 7fbb7fe00640 1 -- 172.21.15.105:0/3687023724 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbb8815cfd0 con 0x7fbb880b8670 2024-03-23T07:57:42.485 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-03-23T07:57:42.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.488+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb6406ac00 msgr2=0x7fbb6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:42.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.488+0000 7fbb8f000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb6406ac00 0x7fbb6406d0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb70036000 tx=0x7fbb70008030 comp rx=0 tx=0).stop 2024-03-23T07:57:42.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.488+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880b8670 msgr2=0x7fbb880c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:42.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.488+0000 7fbb8f000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880b8670 0x7fbb880c8e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c430 tx=0x7fbb7800c8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 shutdown_connections 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb6406ac00 0x7fbb6406d0c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb8815cbf0 0x7fbb8815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb880d2400 0x7fbb881f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 --2- 172.21.15.105:0/3687023724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb880b8670 0x7fbb880c8e70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 >> 172.21.15.105:0/3687023724 conn(0x7fbb880cf560 msgr2=0x7fbb8806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 shutdown_connections 2024-03-23T07:57:42.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.489+0000 7fbb8f000640 1 -- 172.21.15.105:0/3687023724 wait complete. 2024-03-23T07:57:42.510 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T07:57:42.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.714+0000 7fecb4c00640 1 Processor -- start 2024-03-23T07:57:42.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.714+0000 7fecb4c00640 1 -- start start 2024-03-23T07:57:42.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.716+0000 7fecb4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 0x7fecb00cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.716+0000 7fecb4c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb0159c10 con 0x7fecb00ca7b0 2024-03-23T07:57:42.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.716+0000 7fecb4c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb00cd840 con 0x7fecb0159830 2024-03-23T07:57:42.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.716+0000 7fecb4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb006de50 con 0x7fecb00ce930 2024-03-23T07:57:42.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.716+0000 7feca7e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fecb00ca7b0 0x7fecb00cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56036/0 (socket says 172.21.15.105:56036) 2024-03-23T07:57:42.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.716+0000 7feca7e00640 1 -- 172.21.15.105:0/3795234076 learned_addr learned my addr 172.21.15.105:0/3795234076 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:42.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81217861 0 0) 0x7fecb0159c10 con 0x7fecb00ca7b0 2024-03-23T07:57:42.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec94003520 con 0x7fecb00ca7b0 2024-03-23T07:57:42.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2585458499 0 0) 0x7fec94003520 con 0x7fecb00ca7b0 2024-03-23T07:57:42.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 msgr2=0x7fecb00cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:42.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 0x7fecb00cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 >> v1:172.21.15.132:6789/0 conn(0x7fecb0159830 legacy=0x7fecb00cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00ccc30 con 0x7fecb00ca7b0 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fec98001670 con 0x7fecb00ca7b0 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 >> v1:172.21.15.105:6789/0 conn(0x7fecb00ca7b0 legacy=0x7fecb00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.717+0000 7fecad400640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec940039a0 0x7fec94003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecad400640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 0x7fec940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecad400640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec94004e20 0x7fec9400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec94005560 con 0x7fec940039a0 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9400e0d0 con 0x7fec94004e20 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9400e210 con 0x7fecb00ce930 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecade00640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec940039a0 0x7fec94003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecae800640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec94004e20 0x7fec9400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7feca7e00640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 0x7fec940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecade00640 1 -- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 msgr2=0x7fec940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:42.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecade00640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 0x7fec940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecade00640 1 -- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec94004e20 msgr2=0x7fec9400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:42.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecade00640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec94004e20 0x7fec9400db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7fecade00640 1 -- 172.21.15.105:0/3795234076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94003520 con 0x7fec940039a0 2024-03-23T07:57:42.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.718+0000 7feca7e00640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb00ce930 0x7fec940047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.720+0000 7fecade00640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec940039a0 0x7fec94003da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feca0001650 tx=0x7fec940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d23dfc727e3e0d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.720+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0005270 con 0x7fec940039a0 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.720+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca002e070 con 0x7fec940039a0 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.720+0000 7fecad400640 1 -- 172.21.15.105:0/3795234076 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0003610 con 0x7fec940039a0 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.721+0000 7fecb4c00640 1 -- 172.21.15.105:0/3795234076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec940039a0 msgr2=0x7fec94003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.721+0000 7fecb4c00640 1 --2- 172.21.15.105:0/3795234076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec940039a0 0x7fec94003da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feca0001650 tx=0x7fec940057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.721+0000 7fecae800640 1 -- 172.21.15.105:0/3795234076 reap_dead start 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.721+0000 7fecb4c00640 1 -- 172.21.15.105:0/3795234076 shutdown_connections 2024-03-23T07:57:42.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.721+0000 7fecb4c00640 1 -- 172.21.15.105:0/3795234076 >> 172.21.15.105:0/3795234076 conn(0x7fecb006a0b0 msgr2=0x7fecb006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:42.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.721+0000 7fecb4c00640 1 -- 172.21.15.105:0/3795234076 shutdown_connections 2024-03-23T07:57:42.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.721+0000 7fecb4c00640 1 -- 172.21.15.105:0/3795234076 wait complete. 2024-03-23T07:57:42.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.723+0000 7fecb4c00640 1 Processor -- start 2024-03-23T07:57:42.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.723+0000 7fecb4c00640 1 -- start start 2024-03-23T07:57:42.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecb4c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940039a0 0x7fecb015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecb4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb015c8f0 0x7fecb00abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecb4c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb00ac060 0x7fecb00ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecb4c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb014b400 con 0x7fecb00ac060 2024-03-23T07:57:42.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecb4c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01f1db0 con 0x7fec940039a0 2024-03-23T07:57:42.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecb4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00acba0 con 0x7fecb015c8f0 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecae800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb00ac060 0x7fecb00ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecae800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb00ac060 0x7fecb00ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52310/0 (socket says 172.21.15.105:52310) 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecae800640 1 -- 172.21.15.105:0/1528716448 learned_addr learned my addr 172.21.15.105:0/1528716448 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7fecade00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940039a0 0x7fecb015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.724+0000 7feca7e00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb015c8f0 0x7fecb00abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecae800640 1 -- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb015c8f0 msgr2=0x7fecb00abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecae800640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb015c8f0 0x7fecb00abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecae800640 1 -- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940039a0 msgr2=0x7fecb015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecae800640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940039a0 0x7fecb015c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecae800640 1 -- 172.21.15.105:0/1528716448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00ace80 con 0x7fecb00ac060 2024-03-23T07:57:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecae800640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb00ac060 0x7fecb00ac4e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fecb006da60 tx=0x7feca800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7feca6000640 1 -- 172.21.15.105:0/1528716448 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca800be70 con 0x7fecb00ac060 2024-03-23T07:57:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7feca6000640 1 -- 172.21.15.105:0/1528716448 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca8012070 con 0x7fecb00ac060 2024-03-23T07:57:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7feca6000640 1 -- 172.21.15.105:0/1528716448 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8003bb0 con 0x7fecb00ac060 2024-03-23T07:57:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb00ad160 con 0x7fecb00ac060 2024-03-23T07:57:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.725+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb00ae930 con 0x7fecb00ac060 2024-03-23T07:57:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.726+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb00aeb50 con 0x7fecb00ac060 2024-03-23T07:57:42.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.727+0000 7feca6000640 1 -- 172.21.15.105:0/1528716448 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feca800e420 con 0x7fecb00ac060 2024-03-23T07:57:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.728+0000 7feca6000640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec8c06ac00 0x7fec8c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.728+0000 7feca6000640 1 -- 172.21.15.105:0/1528716448 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6195+0+0 (secure 0 0 0) 0x7feca8088bb0 con 0x7fecb00ac060 2024-03-23T07:57:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.728+0000 7fecade00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec8c06ac00 0x7fec8c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.729+0000 7fecade00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec8c06ac00 0x7fec8c06d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feca0033000 tx=0x7feca0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.732+0000 7feca6000640 1 -- 172.21.15.105:0/1528716448 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feca8053b00 con 0x7fecb00ac060 2024-03-23T07:57:42.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:42.936+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecb0073390 con 0x7fecb00ac060 2024-03-23T07:57:43.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.499+0000 7feca6000640 1 -- 172.21.15.105:0/1528716448 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7feca8057120 con 0x7fecb00ac060 2024-03-23T07:57:43.500 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec8c06ac00 msgr2=0x7fec8c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec8c06ac00 0x7fec8c06d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feca0033000 tx=0x7feca0009030 comp rx=0 tx=0).stop 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb00ac060 msgr2=0x7fecb00ac4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb00ac060 0x7fecb00ac4e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fecb006da60 tx=0x7feca800df30 comp rx=0 tx=0).stop 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 shutdown_connections 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb00ac060 0x7fecb00ac4e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fec8c06ac00 0x7fec8c06d0c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecb015c8f0 0x7fecb00abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 --2- 172.21.15.105:0/1528716448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940039a0 0x7fecb015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 >> 172.21.15.105:0/1528716448 conn(0x7fecb006a0b0 msgr2=0x7fecb00ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:43.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.504+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 shutdown_connections 2024-03-23T07:57:43.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.505+0000 7fecb4c00640 1 -- 172.21.15.105:0/1528716448 wait complete. 2024-03-23T07:57:43.531 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-23T07:57:43.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.736+0000 7fbb14a00640 1 Processor -- start 2024-03-23T07:57:43.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.736+0000 7fbb14a00640 1 -- start start 2024-03-23T07:57:43.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb14a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbb100b1c50 0x7fbb100ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:43.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb14a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb100b2030 0x7fbb10069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb14a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb100b33b0 con 0x7fbb100ca840 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb14a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb100b2f90 con 0x7fbb100b2030 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb14a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb10159410 con 0x7fbb100b1c50 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb05e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb100b2030 0x7fbb10069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb0f200640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbb100ca840 0x7fbb100cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56040/0 (socket says 172.21.15.105:56040) 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb05e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb100b2030 0x7fbb10069c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52312/0 (socket says 172.21.15.105:52312) 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.737+0000 7fbb0f200640 1 -- 172.21.15.105:0/339785448 learned_addr learned my addr 172.21.15.105:0/339785448 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.738+0000 7fbb0e800640 1 --2- 172.21.15.105:0/339785448 >> v2:172.21.15.132:3300/0 conn(0x7fbb100b1c50 0x7fbb100ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.738+0000 7fbb05e00640 1 -- 172.21.15.105:0/339785448 >> v2:172.21.15.132:3300/0 conn(0x7fbb100b1c50 msgr2=0x7fbb100ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.738+0000 7fbb05e00640 1 --2- 172.21.15.105:0/339785448 >> v2:172.21.15.132:3300/0 conn(0x7fbb100b1c50 0x7fbb100ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.738+0000 7fbb05e00640 1 -- 172.21.15.105:0/339785448 >> v1:172.21.15.105:6789/0 conn(0x7fbb100ca840 legacy=0x7fbb100cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:43.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.738+0000 7fbb05e00640 1 -- 172.21.15.105:0/339785448 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100b2bd0 con 0x7fbb100b2030 2024-03-23T07:57:43.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.738+0000 7fbb0de00640 1 -- 172.21.15.105:0/339785448 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568523310 0 0) 0x7fbb100b33b0 con 0x7fbb100ca840 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.739+0000 7fbb05e00640 1 --2- 172.21.15.105:0/339785448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb10069c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbb100cace0 tx=0x7fbaf802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea5c1bc4e103653 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.739+0000 7fbb0de00640 1 -- 172.21.15.105:0/339785448 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf802e070 con 0x7fbb100b2030 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.739+0000 7fbb0de00640 1 -- 172.21.15.105:0/339785448 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf80315a0 con 0x7fbb100b2030 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.739+0000 7fbb0de00640 1 -- 172.21.15.105:0/339785448 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8031930 con 0x7fbb100b2030 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 -- 172.21.15.105:0/339785448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 msgr2=0x7fbb10069c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 --2- 172.21.15.105:0/339785448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb10069c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbb100cace0 tx=0x7fbaf802b840 comp rx=0 tx=0).stop 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 -- 172.21.15.105:0/339785448 shutdown_connections 2024-03-23T07:57:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 --2- 172.21.15.105:0/339785448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb10069c90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 --2- 172.21.15.105:0/339785448 >> v2:172.21.15.132:3300/0 conn(0x7fbb100b1c50 0x7fbb100ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 -- 172.21.15.105:0/339785448 >> 172.21.15.105:0/339785448 conn(0x7fbb100b01b0 msgr2=0x7fbb100b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 -- 172.21.15.105:0/339785448 shutdown_connections 2024-03-23T07:57:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.740+0000 7fbb14a00640 1 -- 172.21.15.105:0/339785448 wait complete. 2024-03-23T07:57:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.742+0000 7fbb14a00640 1 Processor -- start 2024-03-23T07:57:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.742+0000 7fbb14a00640 1 -- start start 2024-03-23T07:57:43.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb14a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100b1c50 0x7fbb100ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:43.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb14a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb100c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:43.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb14a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100ca840 0x7fbb100c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:43.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb14a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1006c040 con 0x7fbb100b2030 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb14a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb10151930 con 0x7fbb100b1c50 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb14a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1006c250 con 0x7fbb100ca840 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb05e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb100c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb05e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb100c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52328/0 (socket says 172.21.15.105:52328) 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.743+0000 7fbb05e00640 1 -- 172.21.15.105:0/3010800143 learned_addr learned my addr 172.21.15.105:0/3010800143 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb0e800640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100b1c50 0x7fbb100ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb0f200640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100ca840 0x7fbb100c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb05e00640 1 -- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100ca840 msgr2=0x7fbb100c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb05e00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100ca840 0x7fbb100c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb05e00640 1 -- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100b1c50 msgr2=0x7fbb100ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb05e00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100b1c50 0x7fbb100ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:43.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb05e00640 1 -- 172.21.15.105:0/3010800143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf8009400 con 0x7fbb100b2030 2024-03-23T07:57:43.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.744+0000 7fbb05e00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb100c7650 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009890 tx=0x7fbaf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:43.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.745+0000 7fbb07400640 1 -- 172.21.15.105:0/3010800143 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8032840 con 0x7fbb100b2030 2024-03-23T07:57:43.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.745+0000 7fbb07400640 1 -- 172.21.15.105:0/3010800143 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf80329a0 con 0x7fbb100b2030 2024-03-23T07:57:43.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.745+0000 7fbb07400640 1 -- 172.21.15.105:0/3010800143 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8032d50 con 0x7fbb100b2030 2024-03-23T07:57:43.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.745+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1006c530 con 0x7fbb100b2030 2024-03-23T07:57:43.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.745+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1006caf0 con 0x7fbb100b2030 2024-03-23T07:57:43.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.746+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1004e030 con 0x7fbb100b2030 2024-03-23T07:57:43.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.747+0000 7fbb07400640 1 -- 172.21.15.105:0/3010800143 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbaf803b3f0 con 0x7fbb100b2030 2024-03-23T07:57:43.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.748+0000 7fbb07400640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbaec06ac00 0x7fbaec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:43.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.748+0000 7fbb07400640 1 -- 172.21.15.105:0/3010800143 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5752+0+0 (secure 0 0 0) 0x7fbaf80ab340 con 0x7fbb100b2030 2024-03-23T07:57:43.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.748+0000 7fbb0e800640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbaec06ac00 0x7fbaec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:43.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.749+0000 7fbb0e800640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbaec06ac00 0x7fbaec06d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006fd0 tx=0x7fbb00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:43.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.752+0000 7fbb07400640 1 -- 172.21.15.105:0/3010800143 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbaf80764f0 con 0x7fbb100b2030 2024-03-23T07:57:43.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:43.946+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb1015acc0 con 0x7fbb100b2030 2024-03-23T07:57:44.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.511+0000 7fbb07400640 1 -- 172.21.15.105:0/3010800143 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7fbaf8079b10 con 0x7fbb100b2030 2024-03-23T07:57:44.512 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data_ec' removed 2024-03-23T07:57:44.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.515+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbaec06ac00 msgr2=0x7fbaec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:44.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.515+0000 7fbb14a00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbaec06ac00 0x7fbaec06d0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006fd0 tx=0x7fbb00008040 comp rx=0 tx=0).stop 2024-03-23T07:57:44.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.515+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 msgr2=0x7fbb100c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:44.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.515+0000 7fbb14a00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb100c7650 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009890 tx=0x7fbaf80046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 shutdown_connections 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbaec06ac00 0x7fbaec06d0c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100ca840 0x7fbb100c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100b2030 0x7fbb100c7650 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 --2- 172.21.15.105:0/3010800143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100b1c50 0x7fbb100ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 >> 172.21.15.105:0/3010800143 conn(0x7fbb100b01b0 msgr2=0x7fbb100c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 shutdown_connections 2024-03-23T07:57:44.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.516+0000 7fbb14a00640 1 -- 172.21.15.105:0/3010800143 wait complete. 2024-03-23T07:57:44.538 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T07:57:44.539 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T07:57:44.539 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T07:57:44.539 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:57:44.539 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-23T07:57:44.539 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-23T07:57:44.539 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T07:57:44.539 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:57:44.540 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T07:57:44.540 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T07:57:44.540 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T07:57:44.540 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:57:44.540 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-23T07:57:44.540 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-23T07:57:44.540 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T07:57:44.541 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:57:44.558 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T07:57:44.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-23T07:57:44.563 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T07:57:44.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T07:57:44.566 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T07:57:44.566 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-23T07:57:44.567 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T07:57:44.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T07:57:44.615 INFO:tasks.ceph.mds.d:Started 2024-03-23T07:57:44.615 DEBUG:teuthology.parallel:result is None 2024-03-23T07:57:44.617 INFO:tasks.ceph.mds.c:Started 2024-03-23T07:57:44.617 DEBUG:teuthology.parallel:result is None 2024-03-23T07:57:44.620 INFO:tasks.ceph.mds.b:Started 2024-03-23T07:57:44.620 DEBUG:teuthology.parallel:result is None 2024-03-23T07:57:44.627 INFO:tasks.ceph.mds.a:Started 2024-03-23T07:57:44.627 DEBUG:teuthology.parallel:result is None 2024-03-23T07:57:44.628 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T07:57:44.780 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-03-23T07:57:44.781 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-03-23T07:57:44.781 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-03-23T07:57:44.784 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-03-23T07:57:44.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.820+0000 7fdafe400640 1 Processor -- start 2024-03-23T07:57:44.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.820+0000 7fdafe400640 1 -- start start 2024-03-23T07:57:44.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.821+0000 7fdafe400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdaf815a880 0x7fdaf8099450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:44.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.821+0000 7fdafe400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80ceec0 0x7fdaf80ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:44.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.821+0000 7fdafe400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf8099990 con 0x7fdaf80cf320 2024-03-23T07:57:44.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.821+0000 7fdafe400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80cc5e0 con 0x7fdaf815a880 2024-03-23T07:57:44.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.821+0000 7fdafe400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80cea30 con 0x7fdaf80ceec0 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80ceec0 0x7fdaf80ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf7400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdaf815a880 0x7fdaf8099450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80ceec0 0x7fdaf80ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:58616/0 (socket says 172.21.15.105:58616) 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf6a00640 1 -- 172.21.15.105:0/1537477260 learned_addr learned my addr 172.21.15.105:0/1537477260 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf6a00640 1 -- 172.21.15.105:0/1537477260 >> v2:172.21.15.132:3300/0 conn(0x7fdaf815a880 msgr2=0x7fdaf8099450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf6a00640 1 --2- 172.21.15.105:0/1537477260 >> v2:172.21.15.132:3300/0 conn(0x7fdaf815a880 0x7fdaf8099450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf6a00640 1 -- 172.21.15.105:0/1537477260 >> v1:172.21.15.132:6789/0 conn(0x7fdaf80cf320 legacy=0x7fdaf815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:44.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.822+0000 7fdaf6a00640 1 -- 172.21.15.105:0/1537477260 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8099bc0 con 0x7fdaf80ceec0 2024-03-23T07:57:44.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.823+0000 7fdaf6a00640 1 --2- 172.21.15.105:0/1537477260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80ceec0 0x7fdaf80ad380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009870 tx=0x7fdae402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f75a525cb869e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:44.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdaf6000640 1 -- 172.21.15.105:0/1537477260 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4030840 con 0x7fdaf80ceec0 2024-03-23T07:57:44.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdaf6000640 1 -- 172.21.15.105:0/1537477260 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae40309a0 con 0x7fdaf80ceec0 2024-03-23T07:57:44.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdaf6000640 1 -- 172.21.15.105:0/1537477260 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4030d30 con 0x7fdaf80ceec0 2024-03-23T07:57:44.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 -- 172.21.15.105:0/1537477260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80ceec0 msgr2=0x7fdaf80ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:44.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 --2- 172.21.15.105:0/1537477260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80ceec0 0x7fdaf80ad380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009870 tx=0x7fdae402b980 comp rx=0 tx=0).stop 2024-03-23T07:57:44.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 -- 172.21.15.105:0/1537477260 shutdown_connections 2024-03-23T07:57:44.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 --2- 172.21.15.105:0/1537477260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80ceec0 0x7fdaf80ad380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 --2- 172.21.15.105:0/1537477260 >> v2:172.21.15.132:3300/0 conn(0x7fdaf815a880 0x7fdaf8099450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 -- 172.21.15.105:0/1537477260 >> 172.21.15.105:0/1537477260 conn(0x7fdaf80b2160 msgr2=0x7fdaf80b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:44.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 -- 172.21.15.105:0/1537477260 shutdown_connections 2024-03-23T07:57:44.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.824+0000 7fdafe400640 1 -- 172.21.15.105:0/1537477260 wait complete. 2024-03-23T07:57:44.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.826+0000 7fdafe400640 1 Processor -- start 2024-03-23T07:57:44.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.826+0000 7fdafe400640 1 -- start start 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdafe400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ceec0 0x7fdaf814be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdafe400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80cf320 0x7fdaf80b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdafe400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf815a880 0x7fdaf80aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdafe400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf81f00a0 con 0x7fdaf815a880 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdafe400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80b1f90 con 0x7fdaf80ceec0 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdafe400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf81f02b0 con 0x7fdaf80cf320 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdaf7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf815a880 0x7fdaf80aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:44.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdaf7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf815a880 0x7fdaf80aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52352/0 (socket says 172.21.15.105:52352) 2024-03-23T07:57:44.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.827+0000 7fdaf7e00640 1 -- 172.21.15.105:0/1881635784 learned_addr learned my addr 172.21.15.105:0/1881635784 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:44.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf6a00640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80cf320 0x7fdaf80b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:44.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf7400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ceec0 0x7fdaf814be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:44.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf7e00640 1 -- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80cf320 msgr2=0x7fdaf80b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:44.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf7e00640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80cf320 0x7fdaf80b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf7e00640 1 -- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ceec0 msgr2=0x7fdaf814be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:44.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf7e00640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ceec0 0x7fdaf814be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:44.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf7e00640 1 -- 172.21.15.105:0/1881635784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae40094d0 con 0x7fdaf815a880 2024-03-23T07:57:44.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaf7e00640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf815a880 0x7fdaf80aec90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c400 tx=0x7fdaf000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:44.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaef400640 1 -- 172.21.15.105:0/1881635784 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00052d0 con 0x7fdaf815a880 2024-03-23T07:57:44.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaef400640 1 -- 172.21.15.105:0/1881635784 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0005430 con 0x7fdaf815a880 2024-03-23T07:57:44.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdaef400640 1 -- 172.21.15.105:0/1881635784 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003810 con 0x7fdaf815a880 2024-03-23T07:57:44.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.828+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf81f1030 con 0x7fdaf815a880 2024-03-23T07:57:44.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.829+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf81f0830 con 0x7fdaf815a880 2024-03-23T07:57:44.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.830+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8073390 con 0x7fdaf815a880 2024-03-23T07:57:44.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.831+0000 7fdaef400640 1 -- 172.21.15.105:0/1881635784 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdaf0003970 con 0x7fdaf815a880 2024-03-23T07:57:44.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.832+0000 7fdaef400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdad806ac50 0x7fdad806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:44.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.832+0000 7fdaef400640 1 -- 172.21.15.105:0/1881635784 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fdaf00886b0 con 0x7fdaf815a880 2024-03-23T07:57:44.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.832+0000 7fdaf7400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdad806ac50 0x7fdad806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:44.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.832+0000 7fdaf7400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdad806ac50 0x7fdad806d110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdaf806ea40 tx=0x7fdae800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:44.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:44.836+0000 7fdaef400640 1 -- 172.21.15.105:0/1881635784 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdaf0053ae0 con 0x7fdaf815a880 2024-03-23T07:57:45.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.043+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdaf80b2e60 con 0x7fdaf815a880 2024-03-23T07:57:45.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.814+0000 7fdaef400640 1 -- 172.21.15.105:0/1881635784 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdaf0057100 con 0x7fdaf815a880 2024-03-23T07:57:45.815 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2024-03-23T07:57:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdad806ac50 msgr2=0x7fdad806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdad806ac50 0x7fdad806d110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdaf806ea40 tx=0x7fdae800b040 comp rx=0 tx=0).stop 2024-03-23T07:57:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf815a880 msgr2=0x7fdaf80aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf815a880 0x7fdaf80aec90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c400 tx=0x7fdaf000c8c0 comp rx=0 tx=0).stop 2024-03-23T07:57:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 shutdown_connections 2024-03-23T07:57:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf815a880 0x7fdaf80aec90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdad806ac50 0x7fdad806d110 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80cf320 0x7fdaf80b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 --2- 172.21.15.105:0/1881635784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ceec0 0x7fdaf814be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 >> 172.21.15.105:0/1881635784 conn(0x7fdaf80b2160 msgr2=0x7fdaf81569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:45.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.818+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 shutdown_connections 2024-03-23T07:57:45.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:45.819+0000 7fdafe400640 1 -- 172.21.15.105:0/1881635784 wait complete. 2024-03-23T07:57:45.839 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T07:57:46.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.047+0000 7fb58a600640 1 Processor -- start 2024-03-23T07:57:46.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.048+0000 7fb58a600640 1 -- start start 2024-03-23T07:57:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb58a600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb584098a40 0x7fb58406be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb58a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58415ccd0 0x7fb584098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb58a600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58406c380 con 0x7fb5840cf3c0 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb58a600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb584099360 con 0x7fb584098a40 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb58a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58406ada0 con 0x7fb58415ccd0 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb583400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb584098a40 0x7fb58406be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb583400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb584098a40 0x7fb58406be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52368/0 (socket says 172.21.15.105:52368) 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.049+0000 7fb583400640 1 -- 172.21.15.105:0/1038053806 learned_addr learned my addr 172.21.15.105:0/1038053806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.050+0000 7fb583400640 1 -- 172.21.15.105:0/1038053806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58415ccd0 msgr2=0x7fb584098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.050+0000 7fb583400640 1 --2- 172.21.15.105:0/1038053806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58415ccd0 0x7fb584098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.050+0000 7fb583400640 1 -- 172.21.15.105:0/1038053806 >> v1:172.21.15.132:6789/0 conn(0x7fb5840cf3c0 legacy=0x7fb58406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.050+0000 7fb583400640 1 -- 172.21.15.105:0/1038053806 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58406b080 con 0x7fb584098a40 2024-03-23T07:57:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.051+0000 7fb583400640 1 --2- 172.21.15.105:0/1038053806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb584098a40 0x7fb58406be40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb574009870 tx=0x7fb57402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceadfbacc65329f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.051+0000 7fb582a00640 1 -- 172.21.15.105:0/1038053806 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740317c0 con 0x7fb584098a40 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.051+0000 7fb582a00640 1 -- 172.21.15.105:0/1038053806 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb574031920 con 0x7fb584098a40 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.051+0000 7fb582a00640 1 -- 172.21.15.105:0/1038053806 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574031cb0 con 0x7fb584098a40 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 -- 172.21.15.105:0/1038053806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb584098a40 msgr2=0x7fb58406be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 --2- 172.21.15.105:0/1038053806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb584098a40 0x7fb58406be40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb574009870 tx=0x7fb57402b860 comp rx=0 tx=0).stop 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 -- 172.21.15.105:0/1038053806 shutdown_connections 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 --2- 172.21.15.105:0/1038053806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58415ccd0 0x7fb584098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 --2- 172.21.15.105:0/1038053806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb584098a40 0x7fb58406be40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 -- 172.21.15.105:0/1038053806 >> 172.21.15.105:0/1038053806 conn(0x7fb5840b2160 msgr2=0x7fb5840b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 -- 172.21.15.105:0/1038053806 shutdown_connections 2024-03-23T07:57:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.052+0000 7fb58a600640 1 -- 172.21.15.105:0/1038053806 wait complete. 2024-03-23T07:57:46.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.054+0000 7fb58a600640 1 Processor -- start 2024-03-23T07:57:46.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.054+0000 7fb58a600640 1 -- start start 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.055+0000 7fb58a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb584098a40 0x7fb584150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.055+0000 7fb58a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5840cf3c0 0x7fb5840b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.055+0000 7fb58a600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58415ccd0 0x7fb58415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.055+0000 7fb58a600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840b4680 con 0x7fb58415ccd0 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.055+0000 7fb58a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840b4960 con 0x7fb584098a40 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.055+0000 7fb58a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840b4c40 con 0x7fb5840cf3c0 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58415ccd0 0x7fb58415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58415ccd0 0x7fb58415eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52374/0 (socket says 172.21.15.105:52374) 2024-03-23T07:57:46.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 -- 172.21.15.105:0/2109596239 learned_addr learned my addr 172.21.15.105:0/2109596239 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb57aa00640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5840cf3c0 0x7fb5840b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583400640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb584098a40 0x7fb584150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 -- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5840cf3c0 msgr2=0x7fb5840b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5840cf3c0 0x7fb5840b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 -- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb584098a40 msgr2=0x7fb584150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb584098a40 0x7fb584150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.056+0000 7fb583e00640 1 -- 172.21.15.105:0/2109596239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740094d0 con 0x7fb58415ccd0 2024-03-23T07:57:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.057+0000 7fb583e00640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58415ccd0 0x7fb58415eff0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c780 tx=0x7fb57c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.057+0000 7fb57be00640 1 -- 172.21.15.105:0/2109596239 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c003d80 con 0x7fb58415ccd0 2024-03-23T07:57:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.057+0000 7fb57be00640 1 -- 172.21.15.105:0/2109596239 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c00ce90 con 0x7fb58415ccd0 2024-03-23T07:57:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.057+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5840b4f20 con 0x7fb58415ccd0 2024-03-23T07:57:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.057+0000 7fb57be00640 1 -- 172.21.15.105:0/2109596239 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0052b0 con 0x7fb58415ccd0 2024-03-23T07:57:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.057+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb584150760 con 0x7fb58415ccd0 2024-03-23T07:57:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.058+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5840b2cb0 con 0x7fb58415ccd0 2024-03-23T07:57:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.059+0000 7fb57be00640 1 -- 172.21.15.105:0/2109596239 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb57c005410 con 0x7fb58415ccd0 2024-03-23T07:57:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.060+0000 7fb57be00640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb56006ac00 0x7fb56006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.060+0000 7fb57be00640 1 -- 172.21.15.105:0/2109596239 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb57c088a20 con 0x7fb58415ccd0 2024-03-23T07:57:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.060+0000 7fb583400640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb56006ac00 0x7fb56006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.061+0000 7fb583400640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb56006ac00 0x7fb56006d0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb574036000 tx=0x7fb57403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.065+0000 7fb57be00640 1 -- 172.21.15.105:0/2109596239 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb57c053d80 con 0x7fb58415ccd0 2024-03-23T07:57:46.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.330+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb5840b5440 con 0x7fb58415ccd0 2024-03-23T07:57:46.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.330+0000 7fb57be00640 1 -- 172.21.15.105:0/2109596239 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fb5840b5440 con 0x7fb58415ccd0 2024-03-23T07:57:46.331 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:46.331 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:57:46.331 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2024-03-23T07:57:46.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.331 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDfiv5lS0tvLhAA0B6QpWNT6Gv0jpwz9m7eVQ==", 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDgiv5l4RrwDxAAWuB5mTlINA3nbVSNPnEI4Q==", 2024-03-23T07:57:46.332 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDgiv5lpRAsABAAR7J8huk1V8IaPIYWVVi/Sw==", 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-23T07:57:46.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDgiv5lDEaKIBAAh/0WWw23V91S1a8+nGyfiA==", 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.334 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDliv5ls98rEhAATKBcZ6oqsC7bb1yMNqg/mw==", 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoiv5l9jPMHBAA41vAhx2/0VhbdaCTrsDYKA==", 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.335 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDriv5lyrDIJRAAtgl31leEL9BiyDsScyg3VA==", 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.336 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDuiv5lGjHYLxAAylv6QFtPoGOi+pqqQfbErg==", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2iv5lPwNfAhAAhpGoaSwdTfjRf9MMdV0h0w==", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.337 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD5iv5lmeCiCxAAZSas9f18ILuB/hiBbJ3CoQ==", 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8iv5lkvWfFRAAWLgyFRQb2UBjzt8SM1c4ig==", 2024-03-23T07:57:46.338 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/iv5lqtnCLhAAFQEpnFeggOL+NnTZxRsLHA==", 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.339 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDgiv5lYdvBLRAAvSMYjNTR3seBTqA/6zasaw==", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDhiv5lmXqpBhAAFrxBNDUC6tFroTGFMPq30g==", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-23T07:57:46.340 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDgiv5llbFMNxAAYQEtkulLtGFQzIZWLRQBng==", 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.341 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDhiv5lLq14EBAA9u0DqAzFc2EK9fkAouya/w==", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDeiv5lb+miJBAAo34e/Xvbv25EY4y51E3edQ==", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-23T07:57:46.342 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mds", 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAJi/5ljyadChAAZThpxksbLJCl6bbcfuPcRA==", 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mds" 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mgr", 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAJi/5lOFidChAA9WNKyZO8X9GN6b3k7JSxdQ==", 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.343 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-osd", 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAJi/5lEoOdChAAYRx6FzjGFVg55Xbq/d+5zA==", 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-osd" 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd", 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAJi/5lZ66dChAA+YRcIZzH5H/idG/m3XKIQw==", 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-23T07:57:46.344 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAJi/5lFtmdChAA7Rcpv710uY1sMTQpptwLkg==", 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rgw", 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAJi/5l/fSdChAAjQqOZFCKjNHc92leV66bLQ==", 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDfiv5lYK9uJBAAmspnwmPypBVVNfA6jT1s7A==", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDfiv5lHHKFGBAA1bRoBZNGoRMmDAl1UckSBw==", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-03-23T07:57:46.346 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-23T07:57:46.347 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.347 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:46.347 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-23T07:57:46.347 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb56006ac00 msgr2=0x7fb56006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb56006ac00 0x7fb56006d0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb574036000 tx=0x7fb57403d040 comp rx=0 tx=0).stop 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58415ccd0 msgr2=0x7fb58415eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58415ccd0 0x7fb58415eff0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c780 tx=0x7fb57c00cc40 comp rx=0 tx=0).stop 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 shutdown_connections 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb58415ccd0 0x7fb58415eff0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb56006ac00 0x7fb56006d0c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5840cf3c0 0x7fb5840b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 --2- 172.21.15.105:0/2109596239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb584098a40 0x7fb584150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 >> 172.21.15.105:0/2109596239 conn(0x7fb5840b2160 msgr2=0x7fb5840b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:46.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 shutdown_connections 2024-03-23T07:57:46.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.333+0000 7fb58a600640 1 -- 172.21.15.105:0/2109596239 wait complete. 2024-03-23T07:57:46.352 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-23T07:57:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.562+0000 7fab0be00640 1 Processor -- start 2024-03-23T07:57:46.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.562+0000 7fab0be00640 1 -- start start 2024-03-23T07:57:46.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.563+0000 7fab0be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 0x7fab040cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.564+0000 7fab0be00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0415ae60 con 0x7fab0406c2b0 2024-03-23T07:57:46.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.564+0000 7fab0be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0415b140 con 0x7fab040ce930 2024-03-23T07:57:46.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.564+0000 7fab0be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0415bae0 con 0x7fab040cb660 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.564+0000 7fab09e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fab0406c2b0 0x7fab0406c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56066/0 (socket says 172.21.15.105:56066) 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.564+0000 7fab09e00640 1 -- 172.21.15.105:0/1086835503 learned_addr learned my addr 172.21.15.105:0/1086835503 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.564+0000 7fab08a00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 0x7fab040cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.564+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584294100 0 0) 0x7fab0415ae60 con 0x7fab0406c2b0 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaec003520 con 0x7fab0406c2b0 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1655092775 0 0) 0x7faaec003520 con 0x7fab0406c2b0 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 msgr2=0x7fab040cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 0x7fab040cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 >> v1:172.21.15.132:6789/0 conn(0x7fab040ce930 legacy=0x7fab040ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0415b420 con 0x7fab0406c2b0 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7faafc0019f0 con 0x7fab0406c2b0 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 >> v1:172.21.15.105:6789/0 conn(0x7fab0406c2b0 legacy=0x7fab0406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 0x7faaec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaec003f20 0x7faaec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaec004d00 0x7faaec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0415bae0 con 0x7faaec003f20 2024-03-23T07:57:46.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec00ba60 con 0x7faaec004d00 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaec00bc20 con 0x7fab040cb660 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.565+0000 7fab08a00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaec003f20 0x7faaec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab09e00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaec004d00 0x7faaec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab09400640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 0x7faaec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab08a00640 1 -- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 msgr2=0x7faaec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab08a00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab040cb660 0x7faaec0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab08a00640 1 -- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaec004d00 msgr2=0x7faaec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab08a00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaec004d00 0x7faaec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab08a00640 1 -- 172.21.15.105:0/1086835503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec003520 con 0x7faaec003f20 2024-03-23T07:57:46.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.566+0000 7fab09e00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaec004d00 0x7faaec005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:46.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.567+0000 7fab08a00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaec003f20 0x7faaec0047c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ac0 tx=0x7faaec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9039fa51dbdcb130 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:46.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.567+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf002c6b0 con 0x7faaec003f20 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.567+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf002c810 con 0x7faaec003f20 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.567+0000 7fab03e00640 1 -- 172.21.15.105:0/1086835503 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf002cba0 con 0x7faaec003f20 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.568+0000 7fab0be00640 1 -- 172.21.15.105:0/1086835503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaec003f20 msgr2=0x7faaec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.568+0000 7fab0be00640 1 --2- 172.21.15.105:0/1086835503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaec003f20 0x7faaec0047c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ac0 tx=0x7faaec0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.568+0000 7fab09e00640 1 -- 172.21.15.105:0/1086835503 reap_dead start 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.568+0000 7fab0be00640 1 -- 172.21.15.105:0/1086835503 shutdown_connections 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.568+0000 7fab0be00640 1 -- 172.21.15.105:0/1086835503 >> 172.21.15.105:0/1086835503 conn(0x7fab0406a0b0 msgr2=0x7fab0406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.568+0000 7fab0be00640 1 -- 172.21.15.105:0/1086835503 shutdown_connections 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.568+0000 7fab0be00640 1 -- 172.21.15.105:0/1086835503 wait complete. 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.570+0000 7fab0be00640 1 Processor -- start 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.570+0000 7fab0be00640 1 -- start start 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab0be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaec003f20 0x7fab041f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab0be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab040ad030 0x7fab041f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab0be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab041f28d0 0x7fab04159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab0be00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab041f24b0 con 0x7fab041f28d0 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab0be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab041f5eb0 con 0x7fab040ad030 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab0be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab041f6190 con 0x7faaec003f20 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab041f28d0 0x7fab04159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab041f28d0 0x7fab04159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52398/0 (socket says 172.21.15.105:52398) 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 -- 172.21.15.105:0/1945846451 learned_addr learned my addr 172.21.15.105:0/1945846451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab08a00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab040ad030 0x7fab041f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09400640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaec003f20 0x7fab041f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 -- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaec003f20 msgr2=0x7fab041f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaec003f20 0x7fab041f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 -- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab040ad030 msgr2=0x7fab041f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab040ad030 0x7fab041f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09e00640 1 -- 172.21.15.105:0/1945846451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04155280 con 0x7fab041f28d0 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.571+0000 7fab09400640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaec003f20 0x7fab041f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.572+0000 7fab09e00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab041f28d0 0x7fab04159af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab0406fdc0 tx=0x7faafc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.572+0000 7fab01600640 1 -- 172.21.15.105:0/1945846451 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc004d60 con 0x7fab041f28d0 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.572+0000 7fab01600640 1 -- 172.21.15.105:0/1945846451 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc00bb80 con 0x7fab041f28d0 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.572+0000 7fab01600640 1 -- 172.21.15.105:0/1945846451 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc007110 con 0x7fab041f28d0 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.572+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab041f6470 con 0x7fab041f28d0 2024-03-23T07:57:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.572+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab041f5af0 con 0x7fab041f28d0 2024-03-23T07:57:46.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.573+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab041f25f0 con 0x7fab041f28d0 2024-03-23T07:57:46.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.574+0000 7fab01600640 1 -- 172.21.15.105:0/1945846451 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faafc007270 con 0x7fab041f28d0 2024-03-23T07:57:46.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.575+0000 7fab01600640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7faae406ac30 0x7faae406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:46.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.575+0000 7fab01600640 1 -- 172.21.15.105:0/1945846451 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7faafc088fa0 con 0x7fab041f28d0 2024-03-23T07:57:46.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.575+0000 7fab09400640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7faae406ac30 0x7faae406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:46.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.576+0000 7fab09400640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7faae406ac30 0x7faae406d0f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab040cdef0 tx=0x7faaf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:46.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.580+0000 7fab01600640 1 -- 172.21.15.105:0/1945846451 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fab041f25f0 con 0x7fab041f28d0 2024-03-23T07:57:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.841+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fab041f25f0 con 0x7fab041f28d0 2024-03-23T07:57:46.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.847+0000 7fab01600640 1 -- 172.21.15.105:0/1945846451 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fab041f25f0 con 0x7fab041f28d0 2024-03-23T07:57:46.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7faae406ac30 msgr2=0x7faae406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7faae406ac30 0x7faae406d0f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab040cdef0 tx=0x7faaf8009040 comp rx=0 tx=0).stop 2024-03-23T07:57:46.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab041f28d0 msgr2=0x7fab04159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:46.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab041f28d0 0x7fab04159af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab0406fdc0 tx=0x7faafc00b4a0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 shutdown_connections 2024-03-23T07:57:46.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab041f28d0 0x7fab04159af0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab040ad030 0x7fab041f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7faae406ac30 0x7faae406d0f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 --2- 172.21.15.105:0/1945846451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaec003f20 0x7fab041f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:46.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 >> 172.21.15.105:0/1945846451 conn(0x7fab0406a0b0 msgr2=0x7fab040cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:46.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 shutdown_connections 2024-03-23T07:57:46.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:46.850+0000 7fab0be00640 1 -- 172.21.15.105:0/1945846451 wait complete. 2024-03-23T07:57:46.868 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-23T07:57:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.071+0000 7f198fe00640 1 Processor -- start 2024-03-23T07:57:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.071+0000 7f198fe00640 1 -- start start 2024-03-23T07:57:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1990157180 0x7f199015a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198fe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19900b29d0 0x7f19900b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198fe00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19900b3490 con 0x7f199006d960 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198fe00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199015a1e0 con 0x7f19900b29d0 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19900b3770 con 0x7f1990157180 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198e000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19900b29d0 0x7f19900b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198e000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19900b29d0 0x7f19900b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49422/0 (socket says 172.21.15.105:49422) 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198f400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f199006d960 0x7f19900cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33960/0 (socket says 172.21.15.105:33960) 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198e000640 1 -- 172.21.15.105:0/2060392818 learned_addr learned my addr 172.21.15.105:0/2060392818 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198ea00640 1 --2- 172.21.15.105:0/2060392818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1990157180 0x7f199015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198e000640 1 -- 172.21.15.105:0/2060392818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1990157180 msgr2=0x7f199015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198e000640 1 --2- 172.21.15.105:0/2060392818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1990157180 0x7f199015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.073+0000 7f198e000640 1 -- 172.21.15.105:0/2060392818 >> v1:172.21.15.105:6789/0 conn(0x7f199006d960 legacy=0x7f19900cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.074+0000 7f198e000640 1 -- 172.21.15.105:0/2060392818 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1990159e00 con 0x7f19900b29d0 2024-03-23T07:57:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.074+0000 7f198d600640 1 -- 172.21.15.105:0/2060392818 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664696574 0 0) 0x7f19900b3490 con 0x7f199006d960 2024-03-23T07:57:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.075+0000 7f198e000640 1 --2- 172.21.15.105:0/2060392818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900b29d0 0x7f19900b2e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f197c009870 tx=0x7f197c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fc44e1102fa29 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.075+0000 7f198d600640 1 -- 172.21.15.105:0/2060392818 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c031640 con 0x7f19900b29d0 2024-03-23T07:57:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.075+0000 7f198d600640 1 -- 172.21.15.105:0/2060392818 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c0317a0 con 0x7f19900b29d0 2024-03-23T07:57:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.075+0000 7f198d600640 1 -- 172.21.15.105:0/2060392818 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c031b30 con 0x7f19900b29d0 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 -- 172.21.15.105:0/2060392818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900b29d0 msgr2=0x7f19900b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 --2- 172.21.15.105:0/2060392818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900b29d0 0x7f19900b2e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f197c009870 tx=0x7f197c02b8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 -- 172.21.15.105:0/2060392818 shutdown_connections 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 --2- 172.21.15.105:0/2060392818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19900b29d0 0x7f19900b2e30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 --2- 172.21.15.105:0/2060392818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1990157180 0x7f199015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 -- 172.21.15.105:0/2060392818 >> 172.21.15.105:0/2060392818 conn(0x7f1990065e20 msgr2=0x7f1990069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 -- 172.21.15.105:0/2060392818 shutdown_connections 2024-03-23T07:57:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.076+0000 7f198fe00640 1 -- 172.21.15.105:0/2060392818 wait complete. 2024-03-23T07:57:47.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.077+0000 7f198fe00640 1 Processor -- start 2024-03-23T07:57:47.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.078+0000 7f198fe00640 1 -- start start 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.078+0000 7f198fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199006d960 0x7f199014ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19900b29d0 0x7f199014e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1990157180 0x7f199014b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1990134a50 con 0x7f199006d960 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1990163580 con 0x7f1990157180 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1990163860 con 0x7f19900b29d0 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199006d960 0x7f199014ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199006d960 0x7f199014ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49428/0 (socket says 172.21.15.105:49428) 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 -- 172.21.15.105:0/580579229 learned_addr learned my addr 172.21.15.105:0/580579229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 -- 172.21.15.105:0/580579229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19900b29d0 msgr2=0x7f199014e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19900b29d0 0x7f199014e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 -- 172.21.15.105:0/580579229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1990157180 msgr2=0x7f199014b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1990157180 0x7f199014b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.079+0000 7f198ea00640 1 -- 172.21.15.105:0/580579229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0094d0 con 0x7f199006d960 2024-03-23T07:57:47.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.080+0000 7f198ea00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199006d960 0x7f199014ab50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f198000d5f0 tx=0x7f198000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:47.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.080+0000 7f1986a00640 1 -- 172.21.15.105:0/580579229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800125f0 con 0x7f199006d960 2024-03-23T07:57:47.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.080+0000 7f1986a00640 1 -- 172.21.15.105:0/580579229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980012750 con 0x7f199006d960 2024-03-23T07:57:47.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.080+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1990163b40 con 0x7f199006d960 2024-03-23T07:57:47.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.080+0000 7f1986a00640 1 -- 172.21.15.105:0/580579229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980012b00 con 0x7f199006d960 2024-03-23T07:57:47.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.080+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199014e970 con 0x7f199006d960 2024-03-23T07:57:47.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.081+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199004e030 con 0x7f199006d960 2024-03-23T07:57:47.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.082+0000 7f1986a00640 1 -- 172.21.15.105:0/580579229 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1980012c60 con 0x7f199006d960 2024-03-23T07:57:47.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.083+0000 7f1986a00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f197006ac00 0x7f197006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.083+0000 7f1986a00640 1 -- 172.21.15.105:0/580579229 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f19800895a0 con 0x7f199006d960 2024-03-23T07:57:47.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.083+0000 7f198e000640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f197006ac00 0x7f197006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:47.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.084+0000 7f198e000640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f197006ac00 0x7f197006d0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f197c036000 tx=0x7f197c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:47.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.087+0000 7f1986a00640 1 -- 172.21.15.105:0/580579229 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1980054a20 con 0x7f199006d960 2024-03-23T07:57:47.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.357+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f199006c9e0 con 0x7f199006d960 2024-03-23T07:57:47.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.362+0000 7f1986a00640 1 -- 172.21.15.105:0/580579229 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1980058040 con 0x7f199006d960 2024-03-23T07:57:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f197006ac00 msgr2=0x7f197006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f197006ac00 0x7f197006d0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f197c036000 tx=0x7f197c030000 comp rx=0 tx=0).stop 2024-03-23T07:57:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199006d960 msgr2=0x7f199014ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199006d960 0x7f199014ab50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f198000d5f0 tx=0x7f198000dab0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 shutdown_connections 2024-03-23T07:57:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1990157180 0x7f199014b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f197006ac00 0x7f197006d0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19900b29d0 0x7f199014e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 --2- 172.21.15.105:0/580579229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f199006d960 0x7f199014ab50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 >> 172.21.15.105:0/580579229 conn(0x7f1990065e20 msgr2=0x7f19900708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.365+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 shutdown_connections 2024-03-23T07:57:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.366+0000 7f198fe00640 1 -- 172.21.15.105:0/580579229 wait complete. 2024-03-23T07:57:47.386 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-23T07:57:47.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.586+0000 7f8859e00640 1 Processor -- start 2024-03-23T07:57:47.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.586+0000 7f8859e00640 1 -- start start 2024-03-23T07:57:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.588+0000 7f8859e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88540cd830 0x7f8854160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.588+0000 7f8859e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88541613d0 con 0x7f88540cf7c0 2024-03-23T07:57:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.588+0000 7f8859e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88540ccd90 con 0x7f885415c820 2024-03-23T07:57:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.588+0000 7f8859e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540b7fc0 con 0x7f88540cd830 2024-03-23T07:57:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.589+0000 7f8859400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88540cd830 0x7f8854160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.589+0000 7f884be00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f88540cf7c0 0x7f88540d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33964/0 (socket says 172.21.15.105:33964) 2024-03-23T07:57:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.589+0000 7f884be00640 1 -- 172.21.15.105:0/564237151 learned_addr learned my addr 172.21.15.105:0/564237151 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:47.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.589+0000 7f8859400640 1 -- 172.21.15.105:0/564237151 >> v1:172.21.15.132:6789/0 conn(0x7f885415c820 legacy=0x7f88540cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.589+0000 7f8859400640 1 -- 172.21.15.105:0/564237151 >> v1:172.21.15.105:6789/0 conn(0x7f88540cf7c0 legacy=0x7f88540d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.589+0000 7f8859400640 1 -- 172.21.15.105:0/564237151 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88540b82a0 con 0x7f88540cd830 2024-03-23T07:57:47.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.589+0000 7f8853e00640 1 -- 172.21.15.105:0/564237151 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986651032 0 0) 0x7f883c001110 con 0x7f88540cf7c0 2024-03-23T07:57:47.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.591+0000 7f8859400640 1 --2- 172.21.15.105:0/564237151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cd830 0x7f8854160e90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f884c009d90 tx=0x7f884c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828bac0bb56bef46 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:47.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.591+0000 7f8853e00640 1 -- 172.21.15.105:0/564237151 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c02cd80 con 0x7f88540cd830 2024-03-23T07:57:47.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.591+0000 7f8853e00640 1 -- 172.21.15.105:0/564237151 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c034440 con 0x7f88540cd830 2024-03-23T07:57:47.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.591+0000 7f8853e00640 1 -- 172.21.15.105:0/564237151 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0347d0 con 0x7f88540cd830 2024-03-23T07:57:47.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.592+0000 7f8859e00640 1 -- 172.21.15.105:0/564237151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cd830 msgr2=0x7f8854160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.592+0000 7f8859e00640 1 --2- 172.21.15.105:0/564237151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cd830 0x7f8854160e90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f884c009d90 tx=0x7f884c02bfd0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.592+0000 7f8859e00640 1 -- 172.21.15.105:0/564237151 shutdown_connections 2024-03-23T07:57:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.592+0000 7f8859e00640 1 --2- 172.21.15.105:0/564237151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cd830 0x7f8854160e90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.592+0000 7f8859e00640 1 -- 172.21.15.105:0/564237151 >> 172.21.15.105:0/564237151 conn(0x7f885406b070 msgr2=0x7f885406b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.592+0000 7f8859e00640 1 -- 172.21.15.105:0/564237151 shutdown_connections 2024-03-23T07:57:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.592+0000 7f8859e00640 1 -- 172.21.15.105:0/564237151 wait complete. 2024-03-23T07:57:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.594+0000 7f8859e00640 1 Processor -- start 2024-03-23T07:57:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.594+0000 7f8859e00640 1 -- start start 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f8859e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cd830 0x7f885414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f8859e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cf7c0 0x7f885415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f8859e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885415c820 0x7f88540b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f8859e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b1c40 con 0x7f88540cf7c0 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f8859e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b1e90 con 0x7f88540cd830 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f8859e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b2170 con 0x7f885415c820 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f884be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cf7c0 0x7f885415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.595+0000 7f884be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cf7c0 0x7f885415d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49454/0 (socket says 172.21.15.105:49454) 2024-03-23T07:57:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f884be00640 1 -- 172.21.15.105:0/3644898449 learned_addr learned my addr 172.21.15.105:0/3644898449 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f884be00640 1 -- 172.21.15.105:0/3644898449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885415c820 msgr2=0x7f88540b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f884be00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885415c820 0x7f88540b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f884be00640 1 -- 172.21.15.105:0/3644898449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cd830 msgr2=0x7f885414d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f884be00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cd830 0x7f885414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f884be00640 1 -- 172.21.15.105:0/3644898449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009a70 con 0x7f88540cf7c0 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f884be00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cf7c0 0x7f885415d5c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c780 tx=0x7f883c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f8851600640 1 -- 172.21.15.105:0/3644898449 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0072d0 con 0x7f88540cf7c0 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f8851600640 1 -- 172.21.15.105:0/3644898449 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c007450 con 0x7f88540cf7c0 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f8851600640 1 -- 172.21.15.105:0/3644898449 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c001d10 con 0x7f88540cf7c0 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.596+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88540b2450 con 0x7f88540cf7c0 2024-03-23T07:57:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.597+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88540b2a10 con 0x7f88540cf7c0 2024-03-23T07:57:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.598+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88540b2c30 con 0x7f88540cf7c0 2024-03-23T07:57:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.599+0000 7f8851600640 1 -- 172.21.15.105:0/3644898449 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f883c0047e0 con 0x7f88540cf7c0 2024-03-23T07:57:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.599+0000 7f8851600640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f883006ac00 0x7f883006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.600+0000 7f8851600640 1 -- 172.21.15.105:0/3644898449 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f883c088260 con 0x7f88540cf7c0 2024-03-23T07:57:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.600+0000 7f8858a00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f883006ac00 0x7f883006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:47.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.600+0000 7f8858a00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f883006ac00 0x7f883006d0c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8844000f60 tx=0x7f88440079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:47.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.604+0000 7f8851600640 1 -- 172.21.15.105:0/3644898449 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f883c057060 con 0x7f88540cf7c0 2024-03-23T07:57:47.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.847+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f88541613d0 con 0x7f88540cf7c0 2024-03-23T07:57:47.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.854+0000 7f8851600640 1 -- 172.21.15.105:0/3644898449 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f88541613d0 con 0x7f88540cf7c0 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f883006ac00 msgr2=0x7f883006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f883006ac00 0x7f883006d0c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8844000f60 tx=0x7f88440079b0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cf7c0 msgr2=0x7f885415d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cf7c0 0x7f885415d5c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c780 tx=0x7f883c00cc40 comp rx=0 tx=0).stop 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 shutdown_connections 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f883006ac00 0x7f883006d0c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885415c820 0x7f88540b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88540cf7c0 0x7f885415d5c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 --2- 172.21.15.105:0/3644898449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cd830 0x7f885414d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 >> 172.21.15.105:0/3644898449 conn(0x7f885406b070 msgr2=0x7f88540b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:47.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 shutdown_connections 2024-03-23T07:57:47.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:47.857+0000 7f8859e00640 1 -- 172.21.15.105:0/3644898449 wait complete. 2024-03-23T07:57:47.876 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-23T07:57:48.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.080+0000 7fb1ce000640 1 Processor -- start 2024-03-23T07:57:48.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.080+0000 7fb1ce000640 1 -- start start 2024-03-23T07:57:48.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ce000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1c80cc7c0 0x7fb1c80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ce000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb1c8099150 0x7fb1c815c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ce000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c806b250 con 0x7fb1c806ada0 2024-03-23T07:57:48.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ce000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c8099bc0 con 0x7fb1c80cc7c0 2024-03-23T07:57:48.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ce000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c80cee70 con 0x7fb1c8099150 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ccc00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1c80cc7c0 0x7fb1c80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ccc00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1c80cc7c0 0x7fb1c80ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49460/0 (socket says 172.21.15.105:49460) 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.082+0000 7fb1ccc00640 1 -- 172.21.15.105:0/2668408936 learned_addr learned my addr 172.21.15.105:0/2668408936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.083+0000 7fb1cd600640 1 --2- 172.21.15.105:0/2668408936 >> v2:172.21.15.132:3300/0 conn(0x7fb1c8099150 0x7fb1c815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.083+0000 7fb1ccc00640 1 -- 172.21.15.105:0/2668408936 >> v2:172.21.15.132:3300/0 conn(0x7fb1c8099150 msgr2=0x7fb1c815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.083+0000 7fb1ccc00640 1 --2- 172.21.15.105:0/2668408936 >> v2:172.21.15.132:3300/0 conn(0x7fb1c8099150 0x7fb1c815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.083+0000 7fb1ccc00640 1 -- 172.21.15.105:0/2668408936 >> v1:172.21.15.132:6789/0 conn(0x7fb1c806ada0 legacy=0x7fb1c8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:48.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.083+0000 7fb1ccc00640 1 -- 172.21.15.105:0/2668408936 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c80cf150 con 0x7fb1c80cc7c0 2024-03-23T07:57:48.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.084+0000 7fb1ccc00640 1 --2- 172.21.15.105:0/2668408936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c80cc7c0 0x7fb1c80ce790 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb1c809a010 tx=0x7fb1bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe352340aef45e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:48.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.084+0000 7fb1c7400640 1 -- 172.21.15.105:0/2668408936 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc031870 con 0x7fb1c80cc7c0 2024-03-23T07:57:48.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.084+0000 7fb1c7400640 1 -- 172.21.15.105:0/2668408936 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0319d0 con 0x7fb1c80cc7c0 2024-03-23T07:57:48.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.084+0000 7fb1c7400640 1 -- 172.21.15.105:0/2668408936 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc031d60 con 0x7fb1c80cc7c0 2024-03-23T07:57:48.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 -- 172.21.15.105:0/2668408936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c80cc7c0 msgr2=0x7fb1c80ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 --2- 172.21.15.105:0/2668408936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c80cc7c0 0x7fb1c80ce790 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb1c809a010 tx=0x7fb1bc02b910 comp rx=0 tx=0).stop 2024-03-23T07:57:48.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 -- 172.21.15.105:0/2668408936 shutdown_connections 2024-03-23T07:57:48.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 --2- 172.21.15.105:0/2668408936 >> v2:172.21.15.132:3300/0 conn(0x7fb1c8099150 0x7fb1c815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 --2- 172.21.15.105:0/2668408936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c80cc7c0 0x7fb1c80ce790 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 -- 172.21.15.105:0/2668408936 >> 172.21.15.105:0/2668408936 conn(0x7fb1c80b1fb0 msgr2=0x7fb1c80b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:48.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 -- 172.21.15.105:0/2668408936 shutdown_connections 2024-03-23T07:57:48.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.085+0000 7fb1ce000640 1 -- 172.21.15.105:0/2668408936 wait complete. 2024-03-23T07:57:48.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.087+0000 7fb1ce000640 1 Processor -- start 2024-03-23T07:57:48.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.087+0000 7fb1ce000640 1 -- start start 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1ce000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1c806ada0 0x7fb1c81583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1ce000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c8099150 0x7fb1c815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1ce000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80cc7c0 0x7fb1c815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1ce000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80b0e80 con 0x7fb1c8099150 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1ce000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c8148440 con 0x7fb1c80cc7c0 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1ce000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80b04e0 con 0x7fb1c806ada0 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c8099150 0x7fb1c815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c8099150 0x7fb1c815b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49464/0 (socket says 172.21.15.105:49464) 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1c7e00640 1 -- 172.21.15.105:0/694013372 learned_addr learned my addr 172.21.15.105:0/694013372 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1c7e00640 1 -- 172.21.15.105:0/694013372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1c806ada0 msgr2=0x7fb1c81583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.088+0000 7fb1c7e00640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1c806ada0 0x7fb1c81583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1cd600640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80cc7c0 0x7fb1c815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1c7e00640 1 -- 172.21.15.105:0/694013372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80cc7c0 msgr2=0x7fb1c815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1c7e00640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80cc7c0 0x7fb1c815a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1c7e00640 1 -- 172.21.15.105:0/694013372 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0094d0 con 0x7fb1c8099150 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1c7e00640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c8099150 0x7fb1c815b270 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c430 tx=0x7fb1b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1c4c00640 1 -- 172.21.15.105:0/694013372 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011790 con 0x7fb1c8099150 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1c4c00640 1 -- 172.21.15.105:0/694013372 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b40118f0 con 0x7fb1c8099150 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c80b07c0 con 0x7fb1c8099150 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1c4c00640 1 -- 172.21.15.105:0/694013372 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011ce0 con 0x7fb1c8099150 2024-03-23T07:57:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.089+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c8158a80 con 0x7fb1c8099150 2024-03-23T07:57:48.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.090+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c804e030 con 0x7fb1c8099150 2024-03-23T07:57:48.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.091+0000 7fb1c4c00640 1 -- 172.21.15.105:0/694013372 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb1b401c460 con 0x7fb1c8099150 2024-03-23T07:57:48.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.092+0000 7fb1c4c00640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb1a806ac00 0x7fb1a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.092+0000 7fb1c4c00640 1 -- 172.21.15.105:0/694013372 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb1b40890f0 con 0x7fb1c8099150 2024-03-23T07:57:48.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.092+0000 7fb1ccc00640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb1a806ac00 0x7fb1a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.093+0000 7fb1ccc00640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb1a806ac00 0x7fb1a806d0c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc036000 tx=0x7fb1bc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:48.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.097+0000 7fb1c4c00640 1 -- 172.21.15.105:0/694013372 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb1b40544d0 con 0x7fb1c8099150 2024-03-23T07:57:48.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.364+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fb1c80b2ba0 con 0x7fb1c8099150 2024-03-23T07:57:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.372+0000 7fb1c4c00640 1 -- 172.21.15.105:0/694013372 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb1b4057af0 con 0x7fb1c8099150 2024-03-23T07:57:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb1a806ac00 msgr2=0x7fb1a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb1a806ac00 0x7fb1a806d0c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc036000 tx=0x7fb1bc030000 comp rx=0 tx=0).stop 2024-03-23T07:57:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c8099150 msgr2=0x7fb1c815b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c8099150 0x7fb1c815b270 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c430 tx=0x7fb1b400c8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 shutdown_connections 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c80cc7c0 0x7fb1c815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c8099150 0x7fb1c815b270 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb1a806ac00 0x7fb1a806d0c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 --2- 172.21.15.105:0/694013372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1c806ada0 0x7fb1c81583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.375+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 >> 172.21.15.105:0/694013372 conn(0x7fb1c80b1fb0 msgr2=0x7fb1c80ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.376+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 shutdown_connections 2024-03-23T07:57:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.376+0000 7fb1ce000640 1 -- 172.21.15.105:0/694013372 wait complete. 2024-03-23T07:57:48.395 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-23T07:57:48.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.605+0000 7f977ec00640 1 Processor -- start 2024-03-23T07:57:48.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.605+0000 7f977ec00640 1 -- start start 2024-03-23T07:57:48.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.606+0000 7f977ec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 0x7f977806c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.606+0000 7f977ec00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977815ae60 con 0x7f97780cb660 2024-03-23T07:57:48.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.606+0000 7f977ec00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977815b140 con 0x7f97780ce930 2024-03-23T07:57:48.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.606+0000 7f977ec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977815bae0 con 0x7f97780cbae0 2024-03-23T07:57:48.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9777400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f97780cb660 0x7f977806c2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33976/0 (socket says 172.21.15.105:33976) 2024-03-23T07:57:48.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9777400640 1 -- 172.21.15.105:0/3884263389 learned_addr learned my addr 172.21.15.105:0/3884263389 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:48.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376226142 0 0) 0x7f977815ae60 con 0x7f97780cb660 2024-03-23T07:57:48.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f975c003520 con 0x7f97780cb660 2024-03-23T07:57:48.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f977cc00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 0x7f977806c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2581497082 0 0) 0x7f975c003520 con 0x7f97780cb660 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 msgr2=0x7f977806c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 0x7f977806c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 >> v1:172.21.15.132:6789/0 conn(0x7f97780ce930 legacy=0x7f97780ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977815b420 con 0x7f97780cb660 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.607+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f9760001670 con 0x7f97780cb660 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 >> v1:172.21.15.105:6789/0 conn(0x7f97780cb660 legacy=0x7f977806c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9776a00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 0x7f975c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9776a00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f975c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9776a00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f975c004d00 0x7f975c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977815bae0 con 0x7f975c003fd0 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c00bb40 con 0x7f975c004d00 2024-03-23T07:57:48.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c00bd00 con 0x7f97780cbae0 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777400640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f975c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777400640 1 -- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 msgr2=0x7f975c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777e00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 0x7f975c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f977cc00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f975c004d00 0x7f975c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777400640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 0x7f975c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777400640 1 -- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f975c004d00 msgr2=0x7f975c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777400640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f975c004d00 0x7f975c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777400640 1 -- 172.21.15.105:0/3884263389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c003520 con 0x7f975c003fd0 2024-03-23T07:57:48.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.608+0000 7f9777e00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97780cbae0 0x7f975c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:48.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.609+0000 7f9777400640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f975c0047c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f976000c240 tx=0x7f975c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b56ab93c0dcbfaa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:48.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760005ad0 con 0x7f975c003fd0 2024-03-23T07:57:48.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9760005c30 con 0x7f975c003fd0 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f9776a00640 1 -- 172.21.15.105:0/3884263389 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976002caf0 con 0x7f975c003fd0 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f977ec00640 1 -- 172.21.15.105:0/3884263389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 msgr2=0x7f975c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f977ec00640 1 --2- 172.21.15.105:0/3884263389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f975c0047c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f976000c240 tx=0x7f975c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f977cc00640 1 -- 172.21.15.105:0/3884263389 reap_dead start 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f977ec00640 1 -- 172.21.15.105:0/3884263389 shutdown_connections 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f977ec00640 1 -- 172.21.15.105:0/3884263389 >> 172.21.15.105:0/3884263389 conn(0x7f977806a0b0 msgr2=0x7f977806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f977ec00640 1 -- 172.21.15.105:0/3884263389 shutdown_connections 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.610+0000 7f977ec00640 1 -- 172.21.15.105:0/3884263389 wait complete. 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.612+0000 7f977ec00640 1 Processor -- start 2024-03-23T07:57:48.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.612+0000 7f977ec00640 1 -- start start 2024-03-23T07:57:48.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f977ec00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f97780ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f977ec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9778157110 0x7f97780b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f977ec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97781f1880 0x7f9778151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f977ec00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780afd60 con 0x7f975c003fd0 2024-03-23T07:57:48.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f977ec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778157e50 con 0x7f9778157110 2024-03-23T07:57:48.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f977ec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9778157890 con 0x7f97781f1880 2024-03-23T07:57:48.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f9777e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f97780ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f9777e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f97780ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49492/0 (socket says 172.21.15.105:49492) 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.613+0000 7f9777e00640 1 -- 172.21.15.105:0/2200109968 learned_addr learned my addr 172.21.15.105:0/2200109968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f9777e00640 1 -- 172.21.15.105:0/2200109968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97781f1880 msgr2=0x7f9778151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f9777e00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97781f1880 0x7f9778151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f9777e00640 1 -- 172.21.15.105:0/2200109968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9778157110 msgr2=0x7f97780b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f9777e00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9778157110 0x7f97780b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f9777e00640 1 -- 172.21.15.105:0/2200109968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778155280 con 0x7f975c003fd0 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f9777e00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f97780ae2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f976800d7a0 tx=0x7f976800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f976fe00640 1 -- 172.21.15.105:0/2200109968 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768017070 con 0x7f975c003fd0 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f976fe00640 1 -- 172.21.15.105:0/2200109968 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768005070 con 0x7f975c003fd0 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9778157b70 con 0x7f975c003fd0 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f976fe00640 1 -- 172.21.15.105:0/2200109968 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768005420 con 0x7f975c003fd0 2024-03-23T07:57:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.614+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97780b0290 con 0x7f975c003fd0 2024-03-23T07:57:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.615+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9778073390 con 0x7f975c003fd0 2024-03-23T07:57:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.616+0000 7f976fe00640 1 -- 172.21.15.105:0/2200109968 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9768007030 con 0x7f975c003fd0 2024-03-23T07:57:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.617+0000 7f976fe00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f975406ac30 0x7f975406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.617+0000 7f976fe00640 1 -- 172.21.15.105:0/2200109968 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9768088ad0 con 0x7f975c003fd0 2024-03-23T07:57:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.618+0000 7f9777400640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f975406ac30 0x7f975406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.618+0000 7f9777400640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f975406ac30 0x7f975406d0f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9760034000 tx=0x7f9760007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:48.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.622+0000 7f976fe00640 1 -- 172.21.15.105:0/2200109968 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9768053e30 con 0x7f975c003fd0 2024-03-23T07:57:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.889+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9778070db0 con 0x7f975c003fd0 2024-03-23T07:57:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.897+0000 7f976fe00640 1 -- 172.21.15.105:0/2200109968 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9768057450 con 0x7f975c003fd0 2024-03-23T07:57:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f975406ac30 msgr2=0x7f975406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f975406ac30 0x7f975406d0f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9760034000 tx=0x7f9760007000 comp rx=0 tx=0).stop 2024-03-23T07:57:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 msgr2=0x7f97780ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f97780ae2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f976800d7a0 tx=0x7f976800dc60 comp rx=0 tx=0).stop 2024-03-23T07:57:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 shutdown_connections 2024-03-23T07:57:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f975406ac30 0x7f975406d0f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97781f1880 0x7f9778151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9778157110 0x7f97780b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 --2- 172.21.15.105:0/2200109968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f975c003fd0 0x7f97780ae2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 >> 172.21.15.105:0/2200109968 conn(0x7f977806a0b0 msgr2=0x7f977806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 shutdown_connections 2024-03-23T07:57:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:48.900+0000 7f977ec00640 1 -- 172.21.15.105:0/2200109968 wait complete. 2024-03-23T07:57:48.919 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-23T07:57:49.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.127+0000 7f34f1600640 1 Processor -- start 2024-03-23T07:57:49.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.127+0000 7f34f1600640 1 -- start start 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.128+0000 7f34f1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec159690 0x7f34ec0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.128+0000 7f34f1600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34ec0cc330 0x7f34ec0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34f1600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34ec0b1e10 0x7f34ec0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34f1600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec15e010 con 0x7f34ec0b1e10 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34f1600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec0cc7f0 con 0x7f34ec0cc330 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34f1600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0ccad0 con 0x7f34ec159690 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34ec0b1e10 0x7f34ec0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34ec0b1e10 0x7f34ec0cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49506/0 (socket says 172.21.15.105:49506) 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 -- 172.21.15.105:0/2290254613 learned_addr learned my addr 172.21.15.105:0/2290254613 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 -- 172.21.15.105:0/2290254613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec159690 msgr2=0x7f34ec0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 --2- 172.21.15.105:0/2290254613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec159690 0x7f34ec0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 -- 172.21.15.105:0/2290254613 >> v2:172.21.15.132:3300/0 conn(0x7f34ec0cc330 msgr2=0x7f34ec0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 --2- 172.21.15.105:0/2290254613 >> v2:172.21.15.132:3300/0 conn(0x7f34ec0cc330 0x7f34ec0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.129+0000 7f34eb400640 1 -- 172.21.15.105:0/2290254613 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0ccdb0 con 0x7f34ec0b1e10 2024-03-23T07:57:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.130+0000 7f34eb400640 1 --2- 172.21.15.105:0/2290254613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0b1e10 0x7f34ec0cd890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f34ec06fdb0 tx=0x7f34e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ec535b12f96561 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.131+0000 7f34ea000640 1 -- 172.21.15.105:0/2290254613 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e402ca70 con 0x7f34ec0b1e10 2024-03-23T07:57:49.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.131+0000 7f34ea000640 1 -- 172.21.15.105:0/2290254613 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e402cbd0 con 0x7f34ec0b1e10 2024-03-23T07:57:49.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.131+0000 7f34ea000640 1 -- 172.21.15.105:0/2290254613 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e40343f0 con 0x7f34ec0b1e10 2024-03-23T07:57:49.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.131+0000 7f34f1600640 1 -- 172.21.15.105:0/2290254613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0b1e10 msgr2=0x7f34ec0cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.131+0000 7f34f1600640 1 --2- 172.21.15.105:0/2290254613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0b1e10 0x7f34ec0cd890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f34ec06fdb0 tx=0x7f34e402bed0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.132+0000 7f34f1600640 1 -- 172.21.15.105:0/2290254613 shutdown_connections 2024-03-23T07:57:49.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.132+0000 7f34f1600640 1 --2- 172.21.15.105:0/2290254613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0b1e10 0x7f34ec0cd890 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.132+0000 7f34f1600640 1 --2- 172.21.15.105:0/2290254613 >> v2:172.21.15.132:3300/0 conn(0x7f34ec0cc330 0x7f34ec0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.132+0000 7f34f1600640 1 --2- 172.21.15.105:0/2290254613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec159690 0x7f34ec0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.132+0000 7f34f1600640 1 -- 172.21.15.105:0/2290254613 >> 172.21.15.105:0/2290254613 conn(0x7f34ec06a0b0 msgr2=0x7f34ec06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:49.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.132+0000 7f34f1600640 1 -- 172.21.15.105:0/2290254613 shutdown_connections 2024-03-23T07:57:49.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.132+0000 7f34f1600640 1 -- 172.21.15.105:0/2290254613 wait complete. 2024-03-23T07:57:49.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.133+0000 7f34f1600640 1 Processor -- start 2024-03-23T07:57:49.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.133+0000 7f34f1600640 1 -- start start 2024-03-23T07:57:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.134+0000 7f34f1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec0b1e10 0x7f34ec149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.134+0000 7f34f1600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0cc330 0x7f34ec14d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.134+0000 7f34f1600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34ec159690 0x7f34ec14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.134+0000 7f34f1600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec15ca00 con 0x7f34ec0cc330 2024-03-23T07:57:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.134+0000 7f34f1600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec14e2c0 con 0x7f34ec159690 2024-03-23T07:57:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.134+0000 7f34f1600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec14e5a0 con 0x7f34ec0b1e10 2024-03-23T07:57:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0cc330 0x7f34ec14d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0cc330 0x7f34ec14d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49510/0 (socket says 172.21.15.105:49510) 2024-03-23T07:57:49.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 -- 172.21.15.105:0/2547026469 learned_addr learned my addr 172.21.15.105:0/2547026469 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 -- 172.21.15.105:0/2547026469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec0b1e10 msgr2=0x7f34ec149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec0b1e10 0x7f34ec149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 -- 172.21.15.105:0/2547026469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34ec159690 msgr2=0x7f34ec14dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34ec159690 0x7f34ec14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 -- 172.21.15.105:0/2547026469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4009b30 con 0x7f34ec0cc330 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.135+0000 7f34e3e00640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0cc330 0x7f34ec14d720 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b4a0 tx=0x7f34d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.136+0000 7f34e2a00640 1 -- 172.21.15.105:0/2547026469 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4010610 con 0x7f34ec0cc330 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.136+0000 7f34e2a00640 1 -- 172.21.15.105:0/2547026469 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4010770 con 0x7f34ec0cc330 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.136+0000 7f34e2a00640 1 -- 172.21.15.105:0/2547026469 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4010b20 con 0x7f34ec0cc330 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.136+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec14e880 con 0x7f34ec0cc330 2024-03-23T07:57:49.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.136+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec14ee40 con 0x7f34ec0cc330 2024-03-23T07:57:49.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.137+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec073390 con 0x7f34ec0cc330 2024-03-23T07:57:49.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.138+0000 7f34e2a00640 1 -- 172.21.15.105:0/2547026469 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f34d4010c80 con 0x7f34ec0cc330 2024-03-23T07:57:49.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.139+0000 7f34e2a00640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f34c806ac00 0x7f34c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.139+0000 7f34e2a00640 1 -- 172.21.15.105:0/2547026469 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f34d4088e10 con 0x7f34ec0cc330 2024-03-23T07:57:49.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.139+0000 7f34eaa00640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f34c806ac00 0x7f34c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.140+0000 7f34eaa00640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f34c806ac00 0x7f34c806d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009670 tx=0x7f34dc006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:49.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.143+0000 7f34e2a00640 1 -- 172.21.15.105:0/2547026469 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f34d408c030 con 0x7f34ec0cc330 2024-03-23T07:57:49.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.402+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f34ec0cf230 con 0x7f34ec0cc330 2024-03-23T07:57:49.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.409+0000 7f34e2a00640 1 -- 172.21.15.105:0/2547026469 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f34d40140a0 con 0x7f34ec0cc330 2024-03-23T07:57:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f34c806ac00 msgr2=0x7f34c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f34c806ac00 0x7f34c806d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009670 tx=0x7f34dc006bf0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0cc330 msgr2=0x7f34ec14d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0cc330 0x7f34ec14d720 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b4a0 tx=0x7f34d400b960 comp rx=0 tx=0).stop 2024-03-23T07:57:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 shutdown_connections 2024-03-23T07:57:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34ec159690 0x7f34ec14dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ec0cc330 0x7f34ec14d720 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f34c806ac00 0x7f34c806d0c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 --2- 172.21.15.105:0/2547026469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34ec0b1e10 0x7f34ec149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.411+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 >> 172.21.15.105:0/2547026469 conn(0x7f34ec06a0b0 msgr2=0x7f34ec0ab8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.412+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 shutdown_connections 2024-03-23T07:57:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.412+0000 7f34f1600640 1 -- 172.21.15.105:0/2547026469 wait complete. 2024-03-23T07:57:49.430 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.622+0000 7fe8ace00640 1 Processor -- start 2024-03-23T07:57:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.623+0000 7fe8ace00640 1 -- start start 2024-03-23T07:57:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.624+0000 7fe8ace00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a815b560 0x7fe8a815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.624+0000 7fe8ace00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a815f320 con 0x7fe8a815d410 2024-03-23T07:57:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.624+0000 7fe8ace00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a806c6f0 con 0x7fe8a80b39b0 2024-03-23T07:57:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.624+0000 7fe8ace00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a806c9d0 con 0x7fe8a815b560 2024-03-23T07:57:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a6800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe8a815d410 0x7fe8a815ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33984/0 (socket says 172.21.15.105:33984) 2024-03-23T07:57:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a6800640 1 -- 172.21.15.105:0/2774511050 learned_addr learned my addr 172.21.15.105:0/2774511050 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138561778 0 0) 0x7fe8a815f320 con 0x7fe8a815d410 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe88c003520 con 0x7fe8a815d410 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a815b560 0x7fe8a815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445764480 0 0) 0x7fe88c003520 con 0x7fe8a815d410 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a815b560 msgr2=0x7fe8a815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a815b560 0x7fe8a815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 >> v1:172.21.15.132:6789/0 conn(0x7fe8a80b39b0 legacy=0x7fe8a815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a806ccb0 con 0x7fe8a815d410 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fe8a00019f0 con 0x7fe8a815d410 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 >> v1:172.21.15.105:6789/0 conn(0x7fe8a815d410 legacy=0x7fe8a815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.625+0000 7fe8a5400640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe88c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5400640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe88c0047c0 0x7fe88c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5400640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a815b560 0x7fe88c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a806c9d0 con 0x7fe88c0039a0 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c005500 con 0x7fe88c0047c0 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c00bec0 con 0x7fe8a815b560 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe88c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5e00640 1 -- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a815b560 msgr2=0x7fe88c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a815b560 0x7fe88c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5e00640 1 -- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe88c0047c0 msgr2=0x7fe88c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe88c0047c0 0x7fe88c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.626+0000 7fe8a5e00640 1 -- 172.21.15.105:0/2774511050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c003520 con 0x7fe88c0039a0 2024-03-23T07:57:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.627+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe88c003dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe89800a9a0 tx=0x7fe88c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a220daa1b10d5e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898030660 con 0x7fe88c0039a0 2024-03-23T07:57:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8980307c0 con 0x7fe88c0039a0 2024-03-23T07:57:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8a5400640 1 -- 172.21.15.105:0/2774511050 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898030b70 con 0x7fe88c0039a0 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8ace00640 1 -- 172.21.15.105:0/2774511050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 msgr2=0x7fe88c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8ace00640 1 --2- 172.21.15.105:0/2774511050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe88c003dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe89800a9a0 tx=0x7fe88c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8a6800640 1 -- 172.21.15.105:0/2774511050 reap_dead start 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8ace00640 1 -- 172.21.15.105:0/2774511050 shutdown_connections 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8ace00640 1 -- 172.21.15.105:0/2774511050 >> 172.21.15.105:0/2774511050 conn(0x7fe8a806a3d0 msgr2=0x7fe8a80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8ace00640 1 -- 172.21.15.105:0/2774511050 shutdown_connections 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.628+0000 7fe8ace00640 1 -- 172.21.15.105:0/2774511050 wait complete. 2024-03-23T07:57:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.630+0000 7fe8ace00640 1 Processor -- start 2024-03-23T07:57:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.630+0000 7fe8ace00640 1 -- start start 2024-03-23T07:57:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8ace00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe8a813df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8ace00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a8152fe0 0x7fe8a813f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8ace00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a81523d0 0x7fe8a8152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8ace00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a8152d90 con 0x7fe88c0039a0 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8ace00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a813fa30 con 0x7fe8a8152fe0 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8ace00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a813fcf0 con 0x7fe8a81523d0 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8a5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe8a813df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8a5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe8a813df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49520/0 (socket says 172.21.15.105:49520) 2024-03-23T07:57:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.631+0000 7fe8a5e00640 1 -- 172.21.15.105:0/1769714149 learned_addr learned my addr 172.21.15.105:0/1769714149 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8a5e00640 1 -- 172.21.15.105:0/1769714149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a81523d0 msgr2=0x7fe8a8152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a81523d0 0x7fe8a8152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8a5e00640 1 -- 172.21.15.105:0/1769714149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a8152fe0 msgr2=0x7fe8a813f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe89d400640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a8152fe0 0x7fe8a813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a8152fe0 0x7fe8a813f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8a5e00640 1 -- 172.21.15.105:0/1769714149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a813ffd0 con 0x7fe88c0039a0 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8a5e00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe8a813df40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe89800a120 tx=0x7fe8980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe89ea00640 1 -- 172.21.15.105:0/1769714149 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89803e040 con 0x7fe88c0039a0 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe89ea00640 1 -- 172.21.15.105:0/1769714149 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe898039aa0 con 0x7fe88c0039a0 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe89ea00640 1 -- 172.21.15.105:0/1769714149 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898039e10 con 0x7fe88c0039a0 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a80ae4a0 con 0x7fe88c0039a0 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.632+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a813e480 con 0x7fe88c0039a0 2024-03-23T07:57:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.633+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a813fe30 con 0x7fe88c0039a0 2024-03-23T07:57:49.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.634+0000 7fe89ea00640 1 -- 172.21.15.105:0/1769714149 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe89802f3c0 con 0x7fe88c0039a0 2024-03-23T07:57:49.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.635+0000 7fe89ea00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe88406ac00 0x7fe88406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:49.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.635+0000 7fe89ea00640 1 -- 172.21.15.105:0/1769714149 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe8980ab3f0 con 0x7fe88c0039a0 2024-03-23T07:57:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.635+0000 7fe89d400640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe88406ac00 0x7fe88406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.636+0000 7fe89d400640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe88406ac00 0x7fe88406d0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe8a81542a0 tx=0x7fe890009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.639+0000 7fe89ea00640 1 -- 172.21.15.105:0/1769714149 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe8a813fe30 con 0x7fe88c0039a0 2024-03-23T07:57:49.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.834+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8a813fe30 con 0x7fe88c0039a0 2024-03-23T07:57:49.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.836+0000 7fe89ea00640 1 -- 172.21.15.105:0/1769714149 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12642 (secure 0 0 0) 0x7fe8a813fe30 con 0x7fe88c0039a0 2024-03-23T07:57:49.837 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:49.837 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:57:45.807309+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:49.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.838+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe88406ac00 msgr2=0x7fe88406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.838+0000 7fe8ace00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe88406ac00 0x7fe88406d0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe8a81542a0 tx=0x7fe890009040 comp rx=0 tx=0).stop 2024-03-23T07:57:49.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.838+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 msgr2=0x7fe8a813df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.838+0000 7fe8ace00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe8a813df40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe89800a120 tx=0x7fe8980046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 shutdown_connections 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe88406ac00 0x7fe88406d0c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a81523d0 0x7fe8a8152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a8152fe0 0x7fe8a813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 --2- 172.21.15.105:0/1769714149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe88c0039a0 0x7fe8a813df40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 >> 172.21.15.105:0/1769714149 conn(0x7fe8a806a3d0 msgr2=0x7fe8a80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 shutdown_connections 2024-03-23T07:57:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:49.839+0000 7fe8ace00640 1 -- 172.21.15.105:0/1769714149 wait complete. 2024-03-23T07:57:49.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T07:57:49.858 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:57:50.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.057+0000 7f9ae2600640 1 Processor -- start 2024-03-23T07:57:50.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.058+0000 7f9ae2600640 1 -- start start 2024-03-23T07:57:50.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.059+0000 7f9ae2600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9adc0d0d40 0x7f9adc0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.059+0000 7f9ae2600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc0d20c0 con 0x7f9adc0b84b0 2024-03-23T07:57:50.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.059+0000 7f9ae2600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc0b8940 con 0x7f9adc0cd740 2024-03-23T07:57:50.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.059+0000 7f9ae2600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc0d14b0 con 0x7f9adc0d0d40 2024-03-23T07:57:50.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.059+0000 7f9adaa00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9adc0b84b0 0x7f9adc0b5aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33988/0 (socket says 172.21.15.105:33988) 2024-03-23T07:57:50.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.059+0000 7f9adaa00640 1 -- 172.21.15.105:0/2732142414 learned_addr learned my addr 172.21.15.105:0/2732142414 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:50.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.059+0000 7f9adbe00640 1 --2- 172.21.15.105:0/2732142414 >> v2:172.21.15.105:3300/0 conn(0x7f9adc0d0d40 0x7f9adc0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.060+0000 7f9adbe00640 1 -- 172.21.15.105:0/2732142414 >> v1:172.21.15.132:6789/0 conn(0x7f9adc0cd740 legacy=0x7f9adc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:50.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.060+0000 7f9adbe00640 1 -- 172.21.15.105:0/2732142414 >> v1:172.21.15.105:6789/0 conn(0x7f9adc0b84b0 legacy=0x7f9adc0b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.060+0000 7f9ada000640 1 -- 172.21.15.105:0/2732142414 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589580444 0 0) 0x7f9adc0d20c0 con 0x7f9adc0b84b0 2024-03-23T07:57:50.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.060+0000 7f9adbe00640 1 -- 172.21.15.105:0/2732142414 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0d1790 con 0x7f9adc0d0d40 2024-03-23T07:57:50.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.061+0000 7f9adbe00640 1 --2- 172.21.15.105:0/2732142414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0d0d40 0x7f9adc0b61b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097a0 tx=0x7f9ad402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad3c2a95dd4c7bb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.061+0000 7f9ada000640 1 -- 172.21.15.105:0/2732142414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4030610 con 0x7f9adc0d0d40 2024-03-23T07:57:50.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.061+0000 7f9ada000640 1 -- 172.21.15.105:0/2732142414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4030770 con 0x7f9adc0d0d40 2024-03-23T07:57:50.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.061+0000 7f9ada000640 1 -- 172.21.15.105:0/2732142414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4030b00 con 0x7f9adc0d0d40 2024-03-23T07:57:50.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.062+0000 7f9ae2600640 1 -- 172.21.15.105:0/2732142414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0d0d40 msgr2=0x7f9adc0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.062+0000 7f9ae2600640 1 --2- 172.21.15.105:0/2732142414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0d0d40 0x7f9adc0b61b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097a0 tx=0x7f9ad402b960 comp rx=0 tx=0).stop 2024-03-23T07:57:50.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.062+0000 7f9ae2600640 1 -- 172.21.15.105:0/2732142414 shutdown_connections 2024-03-23T07:57:50.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.062+0000 7f9ae2600640 1 --2- 172.21.15.105:0/2732142414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0d0d40 0x7f9adc0b61b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.062+0000 7f9ae2600640 1 -- 172.21.15.105:0/2732142414 >> 172.21.15.105:0/2732142414 conn(0x7f9adc0cf5d0 msgr2=0x7f9adc0cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:50.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.062+0000 7f9ae2600640 1 -- 172.21.15.105:0/2732142414 shutdown_connections 2024-03-23T07:57:50.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.062+0000 7f9ae2600640 1 -- 172.21.15.105:0/2732142414 wait complete. 2024-03-23T07:57:50.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.064+0000 7f9ae2600640 1 Processor -- start 2024-03-23T07:57:50.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.064+0000 7f9ae2600640 1 -- start start 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9ae2600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9adc0b84b0 0x7f9adc14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9ae2600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0cd740 0x7f9adc0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9ae2600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9adc0d0d40 0x7f9adc0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9ae2600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc150390 con 0x7f9adc0cd740 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9ae2600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0aa270 con 0x7f9adc0b84b0 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9ae2600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0aa550 con 0x7f9adc0d0d40 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9adaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0cd740 0x7f9adc0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9adaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0cd740 0x7f9adc0cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49542/0 (socket says 172.21.15.105:49542) 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9adaa00640 1 -- 172.21.15.105:0/3129168519 learned_addr learned my addr 172.21.15.105:0/3129168519 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9adbe00640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9adc0d0d40 0x7f9adc0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.065+0000 7f9adb400640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9adc0b84b0 0x7f9adc14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adaa00640 1 -- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9adc0d0d40 msgr2=0x7f9adc0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adaa00640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9adc0d0d40 0x7f9adc0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adaa00640 1 -- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9adc0b84b0 msgr2=0x7f9adc14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adaa00640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9adc0b84b0 0x7f9adc14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adaa00640 1 -- 172.21.15.105:0/3129168519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009400 con 0x7f9adc0cd740 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adbe00640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9adc0d0d40 0x7f9adc0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adb400640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9adc0b84b0 0x7f9adc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9adaa00640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0cd740 0x7f9adc0cbb30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c8c0 tx=0x7f9ac400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9ad3400640 1 -- 172.21.15.105:0/3129168519 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400a640 con 0x7f9adc0cd740 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9ad3400640 1 -- 172.21.15.105:0/3129168519 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400a7a0 con 0x7f9adc0cd740 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc0aa830 con 0x7f9adc0cd740 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9ad3400640 1 -- 172.21.15.105:0/3129168519 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400ab90 con 0x7f9adc0cd740 2024-03-23T07:57:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.066+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc0aadf0 con 0x7f9adc0cd740 2024-03-23T07:57:50.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.067+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc0ab010 con 0x7f9adc0cd740 2024-03-23T07:57:50.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.068+0000 7f9ad3400640 1 -- 172.21.15.105:0/3129168519 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ac400acf0 con 0x7f9adc0cd740 2024-03-23T07:57:50.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.069+0000 7f9ad3400640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ab806ac00 0x7f9ab806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.069+0000 7f9ad3400640 1 -- 172.21.15.105:0/3129168519 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9ac4088a30 con 0x7f9adc0cd740 2024-03-23T07:57:50.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.069+0000 7f9adb400640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ab806ac00 0x7f9ab806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.070+0000 7f9adb400640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ab806ac00 0x7f9ab806d0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9acc000f60 tx=0x7f9acc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.073+0000 7f9ad3400640 1 -- 172.21.15.105:0/3129168519 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9ac4053e10 con 0x7f9adc0cd740 2024-03-23T07:57:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.268+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9adc073390 con 0x7f9adc0cd740 2024-03-23T07:57:50.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.269+0000 7f9ad3400640 1 -- 172.21.15.105:0/3129168519 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ac4018020 con 0x7f9adc0cd740 2024-03-23T07:57:50.269 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:57:50.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ab806ac00 msgr2=0x7f9ab806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ab806ac00 0x7f9ab806d0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9acc000f60 tx=0x7f9acc009040 comp rx=0 tx=0).stop 2024-03-23T07:57:50.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0cd740 msgr2=0x7f9adc0cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0cd740 0x7f9adc0cbb30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c8c0 tx=0x7f9ac400cd80 comp rx=0 tx=0).stop 2024-03-23T07:57:50.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 shutdown_connections 2024-03-23T07:57:50.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ab806ac00 0x7f9ab806d0c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9adc0d0d40 0x7f9adc0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9adc0cd740 0x7f9adc0cbb30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 --2- 172.21.15.105:0/3129168519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9adc0b84b0 0x7f9adc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 >> 172.21.15.105:0/3129168519 conn(0x7f9adc0cf5d0 msgr2=0x7f9adc0c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 shutdown_connections 2024-03-23T07:57:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.272+0000 7f9ae2600640 1 -- 172.21.15.105:0/3129168519 wait complete. 2024-03-23T07:57:50.292 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T07:57:50.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.500+0000 7f206a600640 1 Processor -- start 2024-03-23T07:57:50.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.500+0000 7f206a600640 1 -- start start 2024-03-23T07:57:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.501+0000 7f206a600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f206415b560 0x7f206415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.501+0000 7f206a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20640b39b0 0x7f20640b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.501+0000 7f206a600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206406c6f0 con 0x7f206415e500 2024-03-23T07:57:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f206a600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20640b4370 con 0x7f206415b560 2024-03-23T07:57:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f206a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206406c9d0 con 0x7f20640b39b0 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2063400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f206415b560 0x7f206415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2063e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f206415e500 0x7f206415e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33992/0 (socket says 172.21.15.105:33992) 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2063e00640 1 -- 172.21.15.105:0/1698874943 learned_addr learned my addr 172.21.15.105:0/1698874943 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f205aa00640 1 --2- 172.21.15.105:0/1698874943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20640b39b0 0x7f20640b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2063400640 1 -- 172.21.15.105:0/1698874943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20640b39b0 msgr2=0x7f20640b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2063400640 1 --2- 172.21.15.105:0/1698874943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20640b39b0 0x7f20640b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2063400640 1 -- 172.21.15.105:0/1698874943 >> v1:172.21.15.105:6789/0 conn(0x7f206415e500 legacy=0x7f206415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2063400640 1 -- 172.21.15.105:0/1698874943 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206406ccb0 con 0x7f206415b560 2024-03-23T07:57:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.502+0000 7f2062a00640 1 -- 172.21.15.105:0/1698874943 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555048975 0 0) 0x7f206406c6f0 con 0x7f206415e500 2024-03-23T07:57:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.503+0000 7f2063400640 1 --2- 172.21.15.105:0/1698874943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f206415b940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f20540097a0 tx=0x7f205402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba770f0cd336db0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f2062a00640 1 -- 172.21.15.105:0/1698874943 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205402d040 con 0x7f206415b560 2024-03-23T07:57:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f2062a00640 1 -- 172.21.15.105:0/1698874943 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2054031810 con 0x7f206415b560 2024-03-23T07:57:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f2062a00640 1 -- 172.21.15.105:0/1698874943 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054031ba0 con 0x7f206415b560 2024-03-23T07:57:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 -- 172.21.15.105:0/1698874943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 msgr2=0x7f206415b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 --2- 172.21.15.105:0/1698874943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f206415b940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f20540097a0 tx=0x7f205402b8b0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 -- 172.21.15.105:0/1698874943 shutdown_connections 2024-03-23T07:57:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 --2- 172.21.15.105:0/1698874943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20640b39b0 0x7f20640b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 --2- 172.21.15.105:0/1698874943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f206415b940 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 -- 172.21.15.105:0/1698874943 >> 172.21.15.105:0/1698874943 conn(0x7f206406a3d0 msgr2=0x7f20640b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 -- 172.21.15.105:0/1698874943 shutdown_connections 2024-03-23T07:57:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.504+0000 7f206a600640 1 -- 172.21.15.105:0/1698874943 wait complete. 2024-03-23T07:57:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.506+0000 7f206a600640 1 Processor -- start 2024-03-23T07:57:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.506+0000 7f206a600640 1 -- start start 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f206a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20640b39b0 0x7f20641f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f206a600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f2064154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f206a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206415e500 0x7f2064151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f206a600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064153520 con 0x7f206415b560 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f206a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064152910 con 0x7f20640b39b0 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f206a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2064152bf0 con 0x7f206415e500 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f205aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f2064154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.507+0000 7f205aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f2064154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49546/0 (socket says 172.21.15.105:49546) 2024-03-23T07:57:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205aa00640 1 -- 172.21.15.105:0/302691635 learned_addr learned my addr 172.21.15.105:0/302691635 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205aa00640 1 -- 172.21.15.105:0/302691635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206415e500 msgr2=0x7f2064151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205aa00640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206415e500 0x7f2064151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f2063400640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20640b39b0 0x7f20641f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205aa00640 1 -- 172.21.15.105:0/302691635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20640b39b0 msgr2=0x7f20641f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205aa00640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20640b39b0 0x7f20641f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205aa00640 1 -- 172.21.15.105:0/302691635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009400 con 0x7f206415b560 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205aa00640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f2064154260 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f204c007ef0 tx=0x7f204c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205be00640 1 -- 172.21.15.105:0/302691635 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c014620 con 0x7f206415b560 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f205be00640 1 -- 172.21.15.105:0/302691635 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c014780 con 0x7f206415b560 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.508+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2064152ed0 con 0x7f206415b560 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.509+0000 7f205be00640 1 -- 172.21.15.105:0/302691635 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c014b30 con 0x7f206415b560 2024-03-23T07:57:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.509+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641547a0 con 0x7f206415b560 2024-03-23T07:57:50.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.510+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20641536d0 con 0x7f206415b560 2024-03-23T07:57:50.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.511+0000 7f205be00640 1 -- 172.21.15.105:0/302691635 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f204c014c90 con 0x7f206415b560 2024-03-23T07:57:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.511+0000 7f205be00640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f204006ac00 0x7f204006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.512+0000 7f205be00640 1 -- 172.21.15.105:0/302691635 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f204c089b90 con 0x7f206415b560 2024-03-23T07:57:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.512+0000 7f2063400640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f204006ac00 0x7f204006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.512+0000 7f2063400640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f204006ac00 0x7f204006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2054036000 tx=0x7f205403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.516+0000 7f205be00640 1 -- 172.21.15.105:0/302691635 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f20641536d0 con 0x7f206415b560 2024-03-23T07:57:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.709+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f206406dd50 con 0x7f206415b560 2024-03-23T07:57:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.710+0000 7f205be00640 1 -- 172.21.15.105:0/302691635 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+65 (secure 0 0 0) 0x7f204c054ee0 con 0x7f206415b560 2024-03-23T07:57:50.710 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:50.710 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-03-23T07:57:50.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-23T07:57:50.711 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-03-23T07:57:50.711 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-03-23T07:57:50.711 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-23T07:57:50.711 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f204006ac00 msgr2=0x7f204006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f204006ac00 0x7f204006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2054036000 tx=0x7f205403c040 comp rx=0 tx=0).stop 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 msgr2=0x7f2064154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f2064154260 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f204c007ef0 tx=0x7f204c00e4b0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 shutdown_connections 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f204006ac00 0x7f204006d0c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206415e500 0x7f2064151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f206415b560 0x7f2064154260 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 --2- 172.21.15.105:0/302691635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20640b39b0 0x7f20641f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.712+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 >> 172.21.15.105:0/302691635 conn(0x7f206406a3d0 msgr2=0x7f206415c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:50.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.713+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 shutdown_connections 2024-03-23T07:57:50.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.713+0000 7f206a600640 1 -- 172.21.15.105:0/302691635 wait complete. 2024-03-23T07:57:50.729 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T07:57:50.730 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T07:57:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.941+0000 7fb7eca00640 1 Processor -- start 2024-03-23T07:57:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.941+0000 7fb7eca00640 1 -- start start 2024-03-23T07:57:50.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.943+0000 7fb7eca00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb7e80cdde0 0x7fb7e81676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.943+0000 7fb7eca00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e815cfd0 con 0x7fb7e815cbf0 2024-03-23T07:57:50.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.943+0000 7fb7eca00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e80d1100 con 0x7fb7e80d2400 2024-03-23T07:57:50.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.943+0000 7fb7eca00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e80b6a40 con 0x7fb7e80cdde0 2024-03-23T07:57:50.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.943+0000 7fb7e5e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb7e815cbf0 0x7fb7e80d09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:34000/0 (socket says 172.21.15.105:34000) 2024-03-23T07:57:50.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.943+0000 7fb7e5e00640 1 -- 172.21.15.105:0/1740888662 learned_addr learned my addr 172.21.15.105:0/1740888662 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:50.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.943+0000 7fb7e6800640 1 --2- 172.21.15.105:0/1740888662 >> v2:172.21.15.105:3300/0 conn(0x7fb7e80cdde0 0x7fb7e81676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.944+0000 7fb7e4a00640 1 -- 172.21.15.105:0/1740888662 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566113725 0 0) 0x7fb7d8001110 con 0x7fb7e815cbf0 2024-03-23T07:57:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.944+0000 7fb7e4a00640 1 -- 172.21.15.105:0/1740888662 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c8003520 con 0x7fb7e815cbf0 2024-03-23T07:57:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.944+0000 7fb7e6800640 1 -- 172.21.15.105:0/1740888662 >> v1:172.21.15.132:6789/0 conn(0x7fb7e80d2400 legacy=0x7fb7e80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.944+0000 7fb7e4a00640 1 -- 172.21.15.105:0/1740888662 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215817178 0 0) 0x7fb7c8003520 con 0x7fb7e815cbf0 2024-03-23T07:57:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.944+0000 7fb7e6800640 1 -- 172.21.15.105:0/1740888662 >> v1:172.21.15.105:6789/0 conn(0x7fb7e815cbf0 legacy=0x7fb7e80d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.944+0000 7fb7e6800640 1 -- 172.21.15.105:0/1740888662 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80d1360 con 0x7fb7e80cdde0 2024-03-23T07:57:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.945+0000 7fb7e6800640 1 --2- 172.21.15.105:0/1740888662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e80cdde0 0x7fb7e81676b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00097a0 tx=0x7fb7e002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65608a4d34af117d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7e4a00640 1 -- 172.21.15.105:0/1740888662 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8001800 con 0x7fb7e80cdde0 2024-03-23T07:57:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7e4a00640 1 -- 172.21.15.105:0/1740888662 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e0031590 con 0x7fb7e80cdde0 2024-03-23T07:57:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7e4a00640 1 -- 172.21.15.105:0/1740888662 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0031920 con 0x7fb7e80cdde0 2024-03-23T07:57:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7eca00640 1 -- 172.21.15.105:0/1740888662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e80cdde0 msgr2=0x7fb7e81676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7eca00640 1 --2- 172.21.15.105:0/1740888662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e80cdde0 0x7fb7e81676b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00097a0 tx=0x7fb7e002b8b0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7eca00640 1 -- 172.21.15.105:0/1740888662 shutdown_connections 2024-03-23T07:57:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7eca00640 1 --2- 172.21.15.105:0/1740888662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e80cdde0 0x7fb7e81676b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7eca00640 1 -- 172.21.15.105:0/1740888662 >> 172.21.15.105:0/1740888662 conn(0x7fb7e80cf560 msgr2=0x7fb7e80cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7eca00640 1 -- 172.21.15.105:0/1740888662 shutdown_connections 2024-03-23T07:57:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.946+0000 7fb7eca00640 1 -- 172.21.15.105:0/1740888662 wait complete. 2024-03-23T07:57:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.948+0000 7fb7eca00640 1 Processor -- start 2024-03-23T07:57:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.948+0000 7fb7eca00640 1 -- start start 2024-03-23T07:57:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.949+0000 7fb7eca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80cdde0 0x7fb7e8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.949+0000 7fb7eca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7e80d2400 0x7fb7e80c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.949+0000 7fb7eca00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e815cbf0 0x7fb7e80cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.949+0000 7fb7eca00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e815e6d0 con 0x7fb7e815cbf0 2024-03-23T07:57:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.949+0000 7fb7eca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e815dda0 con 0x7fb7e80cdde0 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.949+0000 7fb7eca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e815e080 con 0x7fb7e80d2400 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.949+0000 7fb7e6800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e815cbf0 0x7fb7e80cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e815cbf0 0x7fb7e80cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49562/0 (socket says 172.21.15.105:49562) 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 -- 172.21.15.105:0/1814579994 learned_addr learned my addr 172.21.15.105:0/1814579994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e5400640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7e80d2400 0x7fb7e80c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e5e00640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80cdde0 0x7fb7e8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 -- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7e80d2400 msgr2=0x7fb7e80c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7e80d2400 0x7fb7e80c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 -- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80cdde0 msgr2=0x7fb7e8150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80cdde0 0x7fb7e8150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 -- 172.21.15.105:0/1814579994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009400 con 0x7fb7e815cbf0 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.950+0000 7fb7e6800640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e815cbf0 0x7fb7e80cb6b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00097a0 tx=0x7fb7e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.951+0000 7fb7de000640 1 -- 172.21.15.105:0/1814579994 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0045420 con 0x7fb7e815cbf0 2024-03-23T07:57:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.951+0000 7fb7de000640 1 -- 172.21.15.105:0/1814579994 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e0045580 con 0x7fb7e815cbf0 2024-03-23T07:57:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.951+0000 7fb7eca00640 1 -- 172.21.15.105:0/1814579994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e815e360 con 0x7fb7e815cbf0 2024-03-23T07:57:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.951+0000 7fb7de000640 1 -- 172.21.15.105:0/1814579994 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0045910 con 0x7fb7e815cbf0 2024-03-23T07:57:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.951+0000 7fb7eca00640 1 -- 172.21.15.105:0/1814579994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e815da90 con 0x7fb7e815cbf0 2024-03-23T07:57:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.952+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e80d1bc0 con 0x7fb7e815cbf0 2024-03-23T07:57:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.953+0000 7fb7de000640 1 -- 172.21.15.105:0/1814579994 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb7e0043070 con 0x7fb7e815cbf0 2024-03-23T07:57:50.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.954+0000 7fb7de000640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb7c406ac00 0x7fb7c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:50.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.954+0000 7fb7de000640 1 -- 172.21.15.105:0/1814579994 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb7e00aadf0 con 0x7fb7e815cbf0 2024-03-23T07:57:50.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.954+0000 7fb7e5e00640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb7c406ac00 0x7fb7c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:50.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.955+0000 7fb7e5e00640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb7c406ac00 0x7fb7c406d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8001930 tx=0x7fb7d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:50.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:50.958+0000 7fb7de000640 1 -- 172.21.15.105:0/1814579994 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb7e0076300 con 0x7fb7e815cbf0 2024-03-23T07:57:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.154+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb7e8073390 con 0x7fb7e815cbf0 2024-03-23T07:57:51.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.852+0000 7fb7de000640 1 -- 172.21.15.105:0/1814579994 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb7e0079920 con 0x7fb7e815cbf0 2024-03-23T07:57:51.852 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb7c406ac00 msgr2=0x7fb7c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb7c406ac00 0x7fb7c406d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8001930 tx=0x7fb7d8009040 comp rx=0 tx=0).stop 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e815cbf0 msgr2=0x7fb7e80cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e815cbf0 0x7fb7e80cb6b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00097a0 tx=0x7fb7e00046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 shutdown_connections 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e815cbf0 0x7fb7e80cb6b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb7c406ac00 0x7fb7c406d0c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7e80d2400 0x7fb7e80c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 --2- 172.21.15.105:0/1814579994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80cdde0 0x7fb7e8150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:51.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.856+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 >> 172.21.15.105:0/1814579994 conn(0x7fb7e80cf560 msgr2=0x7fb7e815b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:51.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.857+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 shutdown_connections 2024-03-23T07:57:51.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:51.857+0000 7fb7d7400640 1 -- 172.21.15.105:0/1814579994 wait complete. 2024-03-23T07:57:51.883 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T07:57:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.085+0000 7f3e2aa00640 1 Processor -- start 2024-03-23T07:57:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.085+0000 7f3e2aa00640 1 -- start start 2024-03-23T07:57:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.087+0000 7f3e2aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 0x7f3e240cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.087+0000 7f3e2aa00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e24159c10 con 0x7f3e240ce930 2024-03-23T07:57:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.087+0000 7f3e2aa00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e240cd840 con 0x7f3e240ca7b0 2024-03-23T07:57:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.087+0000 7f3e2aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2406de50 con 0x7f3e24159830 2024-03-23T07:57:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.087+0000 7f3e28a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3e240ce930 0x7f3e240b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:34010/0 (socket says 172.21.15.105:34010) 2024-03-23T07:57:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e28a00640 1 -- 172.21.15.105:0/4023952123 learned_addr learned my addr 172.21.15.105:0/4023952123 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:52.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e23e00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 0x7f3e240cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227744504 0 0) 0x7f3e24159c10 con 0x7f3e240ce930 2024-03-23T07:57:52.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e08003520 con 0x7f3e240ce930 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3628215999 0 0) 0x7f3e08003520 con 0x7f3e240ce930 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 msgr2=0x7f3e240cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 0x7f3e240cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 >> v1:172.21.15.132:6789/0 conn(0x7f3e240ca7b0 legacy=0x7f3e240cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240ccc30 con 0x7f3e240ce930 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.088+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f3e1c0019f0 con 0x7f3e240ce930 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 >> v1:172.21.15.105:6789/0 conn(0x7f3e240ce930 legacy=0x7f3e240b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e22a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 0x7f3e080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e22a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e08003f20 0x7f3e080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e22a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e08004d00 0x7f3e08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0800bac0 con 0x7f3e08004d00 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0800bc30 con 0x7f3e08003f20 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0800e000 con 0x7f3e24159830 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e28a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e08004d00 0x7f3e08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e23400640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e08003f20 0x7f3e080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e23e00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 0x7f3e080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e28a00640 1 -- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 msgr2=0x7f3e080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e28a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e24159830 0x7f3e080039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e28a00640 1 -- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e08003f20 msgr2=0x7f3e080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e28a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e08003f20 0x7f3e080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.089+0000 7f3e28a00640 1 -- 172.21.15.105:0/4023952123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08003520 con 0x7f3e08004d00 2024-03-23T07:57:52.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e28a00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e08004d00 0x7f3e08005180 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00a830 tx=0x7f3e080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4ee5095a057141 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:52.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c034630 con 0x7f3e08004d00 2024-03-23T07:57:52.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c02f070 con 0x7f3e08004d00 2024-03-23T07:57:52.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e22a00640 1 -- 172.21.15.105:0/4023952123 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c0349d0 con 0x7f3e08004d00 2024-03-23T07:57:52.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e2aa00640 1 -- 172.21.15.105:0/4023952123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e08004d00 msgr2=0x7f3e08005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:52.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e2aa00640 1 --2- 172.21.15.105:0/4023952123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e08004d00 0x7f3e08005180 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00a830 tx=0x7f3e080057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e28a00640 1 -- 172.21.15.105:0/4023952123 reap_dead start 2024-03-23T07:57:52.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e2aa00640 1 -- 172.21.15.105:0/4023952123 shutdown_connections 2024-03-23T07:57:52.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e2aa00640 1 -- 172.21.15.105:0/4023952123 >> 172.21.15.105:0/4023952123 conn(0x7f3e2406a0b0 msgr2=0x7f3e2406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:52.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.091+0000 7f3e2aa00640 1 -- 172.21.15.105:0/4023952123 shutdown_connections 2024-03-23T07:57:52.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.092+0000 7f3e2aa00640 1 -- 172.21.15.105:0/4023952123 wait complete. 2024-03-23T07:57:52.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.093+0000 7f3e2aa00640 1 Processor -- start 2024-03-23T07:57:52.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.093+0000 7f3e2aa00640 1 -- start start 2024-03-23T07:57:52.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.094+0000 7f3e2aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e08004d00 0x7f3e24149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.094+0000 7f3e2aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e2415ac20 0x7f3e2415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.094+0000 7f3e2aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e2415b060 0x7f3e240b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.094+0000 7f3e2aa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2415a800 con 0x7f3e2415ac20 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.094+0000 7f3e2aa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2414a3b0 con 0x7f3e2415b060 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.094+0000 7f3e2aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2415bf90 con 0x7f3e08004d00 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e2415ac20 0x7f3e2415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e2415ac20 0x7f3e2415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49584/0 (socket says 172.21.15.105:49584) 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 -- 172.21.15.105:0/134696934 learned_addr learned my addr 172.21.15.105:0/134696934 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e28a00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e2415b060 0x7f3e240b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23e00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e08004d00 0x7f3e24149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 -- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e08004d00 msgr2=0x7f3e24149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e08004d00 0x7f3e24149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 -- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e2415b060 msgr2=0x7f3e240b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:52.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e2415b060 0x7f3e240b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 -- 172.21.15.105:0/134696934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24159e60 con 0x7f3e2415ac20 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.095+0000 7f3e23400640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e2415ac20 0x7f3e2415b510 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000e550 tx=0x7f3e1000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.096+0000 7f3e1be00640 1 -- 172.21.15.105:0/134696934 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e100052e0 con 0x7f3e2415ac20 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.096+0000 7f3e1be00640 1 -- 172.21.15.105:0/134696934 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e10005440 con 0x7f3e2415ac20 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.096+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2415a140 con 0x7f3e2415ac20 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.096+0000 7f3e1be00640 1 -- 172.21.15.105:0/134696934 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003810 con 0x7f3e2415ac20 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.096+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2414a770 con 0x7f3e2415ac20 2024-03-23T07:57:52.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.097+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e24073390 con 0x7f3e2415ac20 2024-03-23T07:57:52.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.098+0000 7f3e1be00640 1 -- 172.21.15.105:0/134696934 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3e10003970 con 0x7f3e2415ac20 2024-03-23T07:57:52.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.099+0000 7f3e1be00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e0406ac30 0x7f3e0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:52.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.100+0000 7f3e1be00640 1 -- 172.21.15.105:0/134696934 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f3e10089080 con 0x7f3e2415ac20 2024-03-23T07:57:52.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.100+0000 7f3e23e00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e0406ac30 0x7f3e0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:52.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.100+0000 7f3e23e00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e0406ac30 0x7f3e0406d0f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009a10 tx=0x7f3e14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:52.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.105+0000 7f3e1be00640 1 -- 172.21.15.105:0/134696934 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3e100bc580 con 0x7f3e2415ac20 2024-03-23T07:57:52.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.297+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3e24158f50 con 0x7f3e2415ac20 2024-03-23T07:57:52.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.872+0000 7f3e1be00640 1 -- 172.21.15.105:0/134696934 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3e10054260 con 0x7f3e2415ac20 2024-03-23T07:57:52.872 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-03-23T07:57:52.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.881+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e0406ac30 msgr2=0x7f3e0406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:52.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.881+0000 7f3e2aa00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e0406ac30 0x7f3e0406d0f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009a10 tx=0x7f3e14008040 comp rx=0 tx=0).stop 2024-03-23T07:57:52.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.881+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e2415ac20 msgr2=0x7f3e2415b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:52.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.881+0000 7f3e2aa00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e2415ac20 0x7f3e2415b510 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000e550 tx=0x7f3e1000ea10 comp rx=0 tx=0).stop 2024-03-23T07:57:52.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.882+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 shutdown_connections 2024-03-23T07:57:52.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.882+0000 7f3e2aa00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e2415b060 0x7f3e240b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.882+0000 7f3e2aa00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e2415ac20 0x7f3e2415b510 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.882+0000 7f3e2aa00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f3e0406ac30 0x7f3e0406d0f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.882+0000 7f3e2aa00640 1 --2- 172.21.15.105:0/134696934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e08004d00 0x7f3e24149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:52.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.882+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 >> 172.21.15.105:0/134696934 conn(0x7f3e2406a0b0 msgr2=0x7f3e24070130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:52.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.884+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 shutdown_connections 2024-03-23T07:57:52.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:52.884+0000 7f3e2aa00640 1 -- 172.21.15.105:0/134696934 wait complete. 2024-03-23T07:57:52.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T07:57:53.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.089+0000 7f50c3a00640 1 Processor -- start 2024-03-23T07:57:53.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.089+0000 7f50c3a00640 1 -- start start 2024-03-23T07:57:53.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50c3a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50bc070150 0x7f50bc0705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50c3a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc070af0 con 0x7f50bc0cc810 2024-03-23T07:57:53.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50c3a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0caec0 con 0x7f50bc158db0 2024-03-23T07:57:53.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50c3a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc0cba90 con 0x7f50bc070150 2024-03-23T07:57:53.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50b3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50bc070150 0x7f50bc0705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50b3e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50bc070150 0x7f50bc0705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49594/0 (socket says 172.21.15.105:49594) 2024-03-23T07:57:53.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50b3e00640 1 -- 172.21.15.105:0/639449520 learned_addr learned my addr 172.21.15.105:0/639449520 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:53.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50c1a00640 1 --1- 172.21.15.105:0/639449520 >> v1:172.21.15.105:6789/0 conn(0x7f50bc0cc810 0x7f50bc0ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:34014/0 (socket says 172.21.15.105:34014) 2024-03-23T07:57:53.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50b3e00640 1 -- 172.21.15.105:0/639449520 >> v1:172.21.15.132:6789/0 conn(0x7f50bc158db0 legacy=0x7f50bc15d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.091+0000 7f50b3e00640 1 -- 172.21.15.105:0/639449520 >> v1:172.21.15.105:6789/0 conn(0x7f50bc0cc810 legacy=0x7f50bc0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.092+0000 7f50b3e00640 1 -- 172.21.15.105:0/639449520 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc159190 con 0x7f50bc070150 2024-03-23T07:57:53.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.092+0000 7f50bbe00640 1 -- 172.21.15.105:0/639449520 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780516575 0 0) 0x7f50bc070af0 con 0x7f50bc0cc810 2024-03-23T07:57:53.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50b3e00640 1 --2- 172.21.15.105:0/639449520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc070150 0x7f50bc0705b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009870 tx=0x7f50a402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c3d1c96cf89a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50bbe00640 1 -- 172.21.15.105:0/639449520 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4030860 con 0x7f50bc070150 2024-03-23T07:57:53.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50bbe00640 1 -- 172.21.15.105:0/639449520 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a40309c0 con 0x7f50bc070150 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50bbe00640 1 -- 172.21.15.105:0/639449520 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4030d50 con 0x7f50bc070150 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50c3a00640 1 -- 172.21.15.105:0/639449520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc070150 msgr2=0x7f50bc0705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50c3a00640 1 --2- 172.21.15.105:0/639449520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc070150 0x7f50bc0705b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009870 tx=0x7f50a402b9a0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50c3a00640 1 -- 172.21.15.105:0/639449520 shutdown_connections 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50c3a00640 1 --2- 172.21.15.105:0/639449520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc070150 0x7f50bc0705b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.093+0000 7f50c3a00640 1 -- 172.21.15.105:0/639449520 >> 172.21.15.105:0/639449520 conn(0x7f50bc06a0b0 msgr2=0x7f50bc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.094+0000 7f50c3a00640 1 -- 172.21.15.105:0/639449520 shutdown_connections 2024-03-23T07:57:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.094+0000 7f50c3a00640 1 -- 172.21.15.105:0/639449520 wait complete. 2024-03-23T07:57:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.095+0000 7f50c3a00640 1 Processor -- start 2024-03-23T07:57:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.095+0000 7f50c3a00640 1 -- start start 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c3a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50bc070150 0x7f50bc145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50bc0cc810 0x7f50bc144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c3a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc158db0 0x7f50bc0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c3a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc140e70 con 0x7f50bc158db0 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c3a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc144560 con 0x7f50bc070150 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1410e0 con 0x7f50bc0cc810 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc158db0 0x7f50bc0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc158db0 0x7f50bc0b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49598/0 (socket says 172.21.15.105:49598) 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.096+0000 7f50c1a00640 1 -- 172.21.15.105:0/82322136 learned_addr learned my addr 172.21.15.105:0/82322136 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c1a00640 1 -- 172.21.15.105:0/82322136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50bc0cc810 msgr2=0x7f50bc144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c1a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50bc0cc810 0x7f50bc144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c1a00640 1 -- 172.21.15.105:0/82322136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50bc070150 msgr2=0x7f50bc145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c1a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50bc070150 0x7f50bc145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c1a00640 1 -- 172.21.15.105:0/82322136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40094d0 con 0x7f50bc158db0 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c1a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc158db0 0x7f50bc0b3630 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f50b400c7e0 tx=0x7f50b400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50b9600640 1 -- 172.21.15.105:0/82322136 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400a640 con 0x7f50bc158db0 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50b9600640 1 -- 172.21.15.105:0/82322136 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b400a7c0 con 0x7f50bc158db0 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50b9600640 1 -- 172.21.15.105:0/82322136 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400ab70 con 0x7f50bc158db0 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc1413c0 con 0x7f50bc158db0 2024-03-23T07:57:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.097+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc141980 con 0x7f50bc158db0 2024-03-23T07:57:53.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.098+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc141b70 con 0x7f50bc158db0 2024-03-23T07:57:53.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.099+0000 7f50b9600640 1 -- 172.21.15.105:0/82322136 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f50b400acd0 con 0x7f50bc158db0 2024-03-23T07:57:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.100+0000 7f50b9600640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f509806ac00 0x7f509806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.100+0000 7f50b9600640 1 -- 172.21.15.105:0/82322136 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f50b4088a30 con 0x7f50bc158db0 2024-03-23T07:57:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.100+0000 7f50c1000640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f509806ac00 0x7f509806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.101+0000 7f50c1000640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f509806ac00 0x7f509806d0c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50ac006fd0 tx=0x7f50ac006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.105+0000 7f50b9600640 1 -- 172.21.15.105:0/82322136 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f50b4053b20 con 0x7f50bc158db0 2024-03-23T07:57:53.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.323+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f50bc073390 con 0x7f50bc158db0 2024-03-23T07:57:53.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.334+0000 7f50b9600640 1 -- 172.21.15.105:0/82322136 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f50b4018070 con 0x7f50bc158db0 2024-03-23T07:57:53.340 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 5 and data pool 6 2024-03-23T07:57:53.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.342+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f509806ac00 msgr2=0x7f509806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.342+0000 7f50c3a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f509806ac00 0x7f509806d0c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50ac006fd0 tx=0x7f50ac006bc0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.342+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc158db0 msgr2=0x7f50bc0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.342+0000 7f50c3a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc158db0 0x7f50bc0b3630 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f50b400c7e0 tx=0x7f50b400cca0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.343+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 shutdown_connections 2024-03-23T07:57:53.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.343+0000 7f50c3a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc158db0 0x7f50bc0b3630 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.343+0000 7f50c3a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f509806ac00 0x7f509806d0c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.343+0000 7f50c3a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50bc0cc810 0x7f50bc144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.343+0000 7f50c3a00640 1 --2- 172.21.15.105:0/82322136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50bc070150 0x7f50bc145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.343+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 >> 172.21.15.105:0/82322136 conn(0x7f50bc06a0b0 msgr2=0x7f50bc070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:53.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.344+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 shutdown_connections 2024-03-23T07:57:53.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.345+0000 7f50c3a00640 1 -- 172.21.15.105:0/82322136 wait complete. 2024-03-23T07:57:53.374 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.540+0000 7f157f800640 1 Processor -- start 2024-03-23T07:57:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.541+0000 7f157f800640 1 -- start start 2024-03-23T07:57:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.542+0000 7f157f800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f15780adbc0 0x7f15780b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.542+0000 7f157f800640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780b16a0 con 0x7f15780cbab0 2024-03-23T07:57:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.542+0000 7f157f800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780b4700 con 0x7f157806d390 2024-03-23T07:57:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.542+0000 7f157f800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15780cbfb0 con 0x7f15780adbc0 2024-03-23T07:57:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.543+0000 7f156fe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f15780cbab0 0x7f15780b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:34016/0 (socket says 172.21.15.105:34016) 2024-03-23T07:57:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.543+0000 7f156fe00640 1 -- 172.21.15.105:0/3911728556 learned_addr learned my addr 172.21.15.105:0/3911728556 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:53.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.543+0000 7f157ce00640 1 --2- 172.21.15.105:0/3911728556 >> v2:172.21.15.132:3300/0 conn(0x7f15780adbc0 0x7f15780b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.543+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629258494 0 0) 0x7f15780b16a0 con 0x7f15780cbab0 2024-03-23T07:57:53.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.543+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f155c003520 con 0x7f15780cbab0 2024-03-23T07:57:53.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996944187 0 0) 0x7f155c003520 con 0x7f15780cbab0 2024-03-23T07:57:53.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 >> v2:172.21.15.132:3300/0 conn(0x7f15780adbc0 msgr2=0x7f15780b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 --2- 172.21.15.105:0/3911728556 >> v2:172.21.15.132:3300/0 conn(0x7f15780adbc0 0x7f15780b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 >> v1:172.21.15.132:6789/0 conn(0x7f157806d390 legacy=0x7f15780b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780cc290 con 0x7f15780cbab0 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f1560001670 con 0x7f15780cbab0 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 >> v1:172.21.15.105:6789/0 conn(0x7f15780cbab0 legacy=0x7f15780b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f155c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155c0047c0 0x7f155c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f155c0051d0 0x7f155c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780cbfb0 con 0x7f155c0039c0 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c00c120 con 0x7f155c0051d0 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c00c2e0 con 0x7f155c0047c0 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f157ce00640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f155c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f157ce00640 1 -- 172.21.15.105:0/3911728556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155c0047c0 msgr2=0x7f155c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f157ce00640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155c0047c0 0x7f155c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f157ce00640 1 -- 172.21.15.105:0/3911728556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f155c0051d0 msgr2=0x7f155c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f157ce00640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f155c0051d0 0x7f155c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f157ce00640 1 -- 172.21.15.105:0/3911728556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c003520 con 0x7f155c0039c0 2024-03-23T07:57:53.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.544+0000 7f157d800640 1 -- 172.21.15.105:0/3911728556 reap_dead start 2024-03-23T07:57:53.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.545+0000 7f157ce00640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f155c003de0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1568009a90 tx=0x7f155c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512f3b4522fccad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.545+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156802c6b0 con 0x7f155c0039c0 2024-03-23T07:57:53.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156802c810 con 0x7f155c0039c0 2024-03-23T07:57:53.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f1577e00640 1 -- 172.21.15.105:0/3911728556 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156802cba0 con 0x7f155c0039c0 2024-03-23T07:57:53.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f157f800640 1 -- 172.21.15.105:0/3911728556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 msgr2=0x7f155c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f157f800640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f155c003de0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1568009a90 tx=0x7f155c0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f157f800640 1 -- 172.21.15.105:0/3911728556 shutdown_connections 2024-03-23T07:57:53.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f157f800640 1 --2- 172.21.15.105:0/3911728556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f155c003de0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f157f800640 1 -- 172.21.15.105:0/3911728556 >> 172.21.15.105:0/3911728556 conn(0x7f157806a3d0 msgr2=0x7f15780b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:53.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f157f800640 1 -- 172.21.15.105:0/3911728556 shutdown_connections 2024-03-23T07:57:53.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.546+0000 7f157f800640 1 -- 172.21.15.105:0/3911728556 wait complete. 2024-03-23T07:57:53.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.548+0000 7f157f800640 1 Processor -- start 2024-03-23T07:57:53.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.548+0000 7f157f800640 1 -- start start 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157f800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f1578136c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157f800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578135d70 0x7f15781591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157f800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f157815c100 0x7f157815c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157f800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157813d270 con 0x7f155c0039c0 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157f800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157815cac0 con 0x7f157815c100 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157f800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157815e360 con 0x7f1578135d70 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f1578136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f1578136c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49622/0 (socket says 172.21.15.105:49622) 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 -- 172.21.15.105:0/3450264969 learned_addr learned my addr 172.21.15.105:0/3450264969 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 -- 172.21.15.105:0/3450264969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578135d70 msgr2=0x7f15781591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f156fe00640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578135d70 0x7f15781591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578135d70 0x7f15781591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 -- 172.21.15.105:0/3450264969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f157815c100 msgr2=0x7f157815c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f157815c100 0x7f157815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.549+0000 7f157ce00640 1 -- 172.21.15.105:0/3450264969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1578137300 con 0x7f155c0039c0 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.550+0000 7f157ce00640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f1578136c60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1568009a90 tx=0x7f1568004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.550+0000 7f1575600640 1 -- 172.21.15.105:0/3450264969 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568003810 con 0x7f155c0039c0 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.550+0000 7f1575600640 1 -- 172.21.15.105:0/3450264969 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1568003970 con 0x7f155c0039c0 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.550+0000 7f1575600640 1 -- 172.21.15.105:0/3450264969 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568003d00 con 0x7f155c0039c0 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.550+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15781375e0 con 0x7f155c0039c0 2024-03-23T07:57:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.550+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1578137ba0 con 0x7f155c0039c0 2024-03-23T07:57:53.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.551+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157806ac20 con 0x7f155c0039c0 2024-03-23T07:57:53.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.552+0000 7f1575600640 1 -- 172.21.15.105:0/3450264969 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1568040450 con 0x7f155c0039c0 2024-03-23T07:57:53.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.553+0000 7f1575600640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac80 0x7f155406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.553+0000 7f1575600640 1 -- 172.21.15.105:0/3450264969 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15680ab1d0 con 0x7f155c0039c0 2024-03-23T07:57:53.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.553+0000 7f156fe00640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac80 0x7f155406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.554+0000 7f156fe00640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac80 0x7f155406d140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f157815e540 tx=0x7f1560009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.558+0000 7f1575600640 1 -- 172.21.15.105:0/3450264969 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f15680763e0 con 0x7f155c0039c0 2024-03-23T07:57:53.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.744+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1578073390 con 0x7f155c0039c0 2024-03-23T07:57:53.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.746+0000 7f1575600640 1 -- 172.21.15.105:0/3450264969 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f15680044e0 con 0x7f155c0039c0 2024-03-23T07:57:53.745 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:53.746 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:57:53.325051+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:57:51.204124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-23T07:57:52.349446+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac80 msgr2=0x7f155406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac80 0x7f155406d140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f157815e540 tx=0x7f1560009040 comp rx=0 tx=0).stop 2024-03-23T07:57:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 msgr2=0x7f1578136c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f1578136c60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1568009a90 tx=0x7f1568004730 comp rx=0 tx=0).stop 2024-03-23T07:57:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 shutdown_connections 2024-03-23T07:57:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f157815c100 0x7f157815c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac80 0x7f155406d140 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578135d70 0x7f15781591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 --2- 172.21.15.105:0/3450264969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f155c0039c0 0x7f1578136c60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 >> 172.21.15.105:0/3450264969 conn(0x7f157806a3d0 msgr2=0x7f15780b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.748+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 shutdown_connections 2024-03-23T07:57:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.749+0000 7f157f800640 1 -- 172.21.15.105:0/3450264969 wait complete. 2024-03-23T07:57:53.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:53.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.965+0000 7f13a3000640 1 Processor -- start 2024-03-23T07:57:53.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.965+0000 7f13a3000640 1 -- start start 2024-03-23T07:57:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.967+0000 7f13a3000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f139c0cf5c0 0x7f139c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.967+0000 7f13a3000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c15ce90 con 0x7f139c15cab0 2024-03-23T07:57:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.967+0000 7f13a3000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c0cddf0 con 0x7f139c0d0c20 2024-03-23T07:57:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.967+0000 7f13a3000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139c0cd570 con 0x7f139c0cf5c0 2024-03-23T07:57:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.967+0000 7f13a1c00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f139c15cab0 0x7f139c0ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:34030/0 (socket says 172.21.15.105:34030) 2024-03-23T07:57:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.967+0000 7f13a1c00640 1 -- 172.21.15.105:0/3564225026 learned_addr learned my addr 172.21.15.105:0/3564225026 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:53.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269881294 0 0) 0x7f139c15ce90 con 0x7f139c15cab0 2024-03-23T07:57:53.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1384003520 con 0x7f139c15cab0 2024-03-23T07:57:53.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f13a1200640 1 --2- 172.21.15.105:0/3564225026 >> v2:172.21.15.132:3300/0 conn(0x7f139c0cf5c0 0x7f139c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 373957632 0 0) 0x7f1384003520 con 0x7f139c15cab0 2024-03-23T07:57:53.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 >> v2:172.21.15.132:3300/0 conn(0x7f139c0cf5c0 msgr2=0x7f139c0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 --2- 172.21.15.105:0/3564225026 >> v2:172.21.15.132:3300/0 conn(0x7f139c0cf5c0 0x7f139c0b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 >> v1:172.21.15.132:6789/0 conn(0x7f139c0d0c20 legacy=0x7f139c0d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c161370 con 0x7f139c15cab0 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f1390001670 con 0x7f139c15cab0 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 >> v1:172.21.15.105:6789/0 conn(0x7f139c15cab0 legacy=0x7f139c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f1384003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13840047c0 0x7f1384004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1384005180 0x7f138400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c0cd570 con 0x7f1384003970 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138400c020 con 0x7f13840047c0 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.968+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138400c1e0 con 0x7f1384005180 2024-03-23T07:57:53.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.969+0000 7f13a1c00640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f1384003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.969+0000 7f13a1c00640 1 -- 172.21.15.105:0/3564225026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1384005180 msgr2=0x7f138400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.969+0000 7f13a1c00640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1384005180 0x7f138400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.969+0000 7f13a1c00640 1 -- 172.21.15.105:0/3564225026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13840047c0 msgr2=0x7f1384004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:53.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.969+0000 7f13a1c00640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13840047c0 0x7f1384004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.969+0000 7f13a1c00640 1 -- 172.21.15.105:0/3564225026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384003520 con 0x7f1384003970 2024-03-23T07:57:53.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.969+0000 7f13a2600640 1 -- 172.21.15.105:0/3564225026 reap_dead start 2024-03-23T07:57:53.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.970+0000 7f13a1c00640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f1384003d90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f139000c240 tx=0x7f13840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f086d7cc8790bce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.970+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390005ad0 con 0x7f1384003970 2024-03-23T07:57:53.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.970+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1390005c30 con 0x7f1384003970 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.970+0000 7f139be00640 1 -- 172.21.15.105:0/3564225026 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139002caf0 con 0x7f1384003970 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.971+0000 7f13a3000640 1 -- 172.21.15.105:0/3564225026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 msgr2=0x7f1384003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.971+0000 7f13a3000640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f1384003d90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f139000c240 tx=0x7f13840057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.971+0000 7f13a3000640 1 -- 172.21.15.105:0/3564225026 shutdown_connections 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.971+0000 7f13a3000640 1 --2- 172.21.15.105:0/3564225026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f1384003d90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.971+0000 7f13a3000640 1 -- 172.21.15.105:0/3564225026 >> 172.21.15.105:0/3564225026 conn(0x7f139c06ae90 msgr2=0x7f139c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.971+0000 7f13a3000640 1 -- 172.21.15.105:0/3564225026 shutdown_connections 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.971+0000 7f13a3000640 1 -- 172.21.15.105:0/3564225026 wait complete. 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.973+0000 7f13a3000640 1 Processor -- start 2024-03-23T07:57:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.973+0000 7f13a3000640 1 -- start start 2024-03-23T07:57:53.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a3000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f139c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a3000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c154d80 0x7f139c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a3000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f139c15d180 0x7f139c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a3000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c15f400 con 0x7f1384003970 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a3000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c157830 con 0x7f139c154d80 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a3000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139c15e780 con 0x7f139c15d180 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f139c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f139c15e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49634/0 (socket says 172.21.15.105:49634) 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 -- 172.21.15.105:0/3511393555 learned_addr learned my addr 172.21.15.105:0/3511393555 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 -- 172.21.15.105:0/3511393555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f139c15d180 msgr2=0x7f139c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f139c15d180 0x7f139c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 -- 172.21.15.105:0/3511393555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c154d80 msgr2=0x7f139c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c154d80 0x7f139c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:53.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.974+0000 7f13a1c00640 1 -- 172.21.15.105:0/3511393555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c15ea60 con 0x7f1384003970 2024-03-23T07:57:53.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.975+0000 7f13a1c00640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f139c15e0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1390001930 tx=0x7f139002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.975+0000 7f1399600640 1 -- 172.21.15.105:0/3511393555 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139003c040 con 0x7f1384003970 2024-03-23T07:57:53.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.975+0000 7f1399600640 1 -- 172.21.15.105:0/3511393555 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139002ce00 con 0x7f1384003970 2024-03-23T07:57:53.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.975+0000 7f1399600640 1 -- 172.21.15.105:0/3511393555 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13900072b0 con 0x7f1384003970 2024-03-23T07:57:53.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.975+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c15ed40 con 0x7f1384003970 2024-03-23T07:57:53.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.975+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c0b28f0 con 0x7f1384003970 2024-03-23T07:57:53.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.976+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139c04e030 con 0x7f1384003970 2024-03-23T07:57:53.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.977+0000 7f1399600640 1 -- 172.21.15.105:0/3511393555 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1390032070 con 0x7f1384003970 2024-03-23T07:57:53.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.978+0000 7f1399600640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f137c06ac30 0x7f137c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:53.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.978+0000 7f1399600640 1 -- 172.21.15.105:0/3511393555 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f13900aaa80 con 0x7f1384003970 2024-03-23T07:57:53.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.978+0000 7f13a1200640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f137c06ac30 0x7f137c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:53.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.979+0000 7f13a1200640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f137c06ac30 0x7f137c06d0f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13880093d0 tx=0x7f1388009300 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:53.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:53.982+0000 7f1399600640 1 -- 172.21.15.105:0/3511393555 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1390075b20 con 0x7f1384003970 2024-03-23T07:57:54.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.176+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f139c0cc970 con 0x7f1384003970 2024-03-23T07:57:54.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.178+0000 7f1399600640 1 -- 172.21.15.105:0/3511393555 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f1390079140 con 0x7f1384003970 2024-03-23T07:57:54.178 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:54.179 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:57:53.325051+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:57:51.204124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-23T07:57:52.349446+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:54.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f137c06ac30 msgr2=0x7f137c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:54.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f137c06ac30 0x7f137c06d0f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13880093d0 tx=0x7f1388009300 comp rx=0 tx=0).stop 2024-03-23T07:57:54.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 msgr2=0x7f139c15e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:54.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f139c15e0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1390001930 tx=0x7f139002e070 comp rx=0 tx=0).stop 2024-03-23T07:57:54.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 shutdown_connections 2024-03-23T07:57:54.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f137c06ac30 0x7f137c06d0f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f139c15d180 0x7f139c156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f139c154d80 0x7f139c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 --2- 172.21.15.105:0/3511393555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1384003970 0x7f139c15e0a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.181+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 >> 172.21.15.105:0/3511393555 conn(0x7f139c06ae90 msgr2=0x7f139c0b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:54.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.182+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 shutdown_connections 2024-03-23T07:57:54.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.182+0000 7f13a3000640 1 -- 172.21.15.105:0/3511393555 wait complete. 2024-03-23T07:57:54.199 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T07:57:54.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.407+0000 7ff396000640 1 Processor -- start 2024-03-23T07:57:54.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.407+0000 7ff396000640 1 -- start start 2024-03-23T07:57:54.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.408+0000 7ff396000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff390160d20 0x7ff390164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:54.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.408+0000 7ff396000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900cf040 0x7ff3900b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:54.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.408+0000 7ff396000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff390165830 con 0x7ff3900ca7b0 2024-03-23T07:57:54.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.408+0000 7ff396000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff390164f90 con 0x7ff390160d20 2024-03-23T07:57:54.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.408+0000 7ff396000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff390165270 con 0x7ff3900cf040 2024-03-23T07:57:54.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.409+0000 7ff38fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900cf040 0x7ff3900b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:54.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.409+0000 7ff38fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900cf040 0x7ff3900b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:55504/0 (socket says 172.21.15.105:55504) 2024-03-23T07:57:54.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.409+0000 7ff38fe00640 1 -- 172.21.15.105:0/253440514 learned_addr learned my addr 172.21.15.105:0/253440514 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:54.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.409+0000 7ff38f400640 1 --2- 172.21.15.105:0/253440514 >> v2:172.21.15.132:3300/0 conn(0x7ff390160d20 0x7ff390164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:54.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.410+0000 7ff38fe00640 1 -- 172.21.15.105:0/253440514 >> v2:172.21.15.132:3300/0 conn(0x7ff390160d20 msgr2=0x7ff390164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:54.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.410+0000 7ff38fe00640 1 --2- 172.21.15.105:0/253440514 >> v2:172.21.15.132:3300/0 conn(0x7ff390160d20 0x7ff390164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.410+0000 7ff38fe00640 1 -- 172.21.15.105:0/253440514 >> v1:172.21.15.132:6789/0 conn(0x7ff3900ca7b0 legacy=0x7ff3900ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:54.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.410+0000 7ff38fe00640 1 -- 172.21.15.105:0/253440514 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390165550 con 0x7ff3900cf040 2024-03-23T07:57:54.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.411+0000 7ff38fe00640 1 --2- 172.21.15.105:0/253440514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900cf040 0x7ff3900b1b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff388009e50 tx=0x7ff38802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64492da1ef82c5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:54.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.412+0000 7ff38ea00640 1 -- 172.21.15.105:0/253440514 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38802ca70 con 0x7ff3900cf040 2024-03-23T07:57:54.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.412+0000 7ff38ea00640 1 -- 172.21.15.105:0/253440514 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38802cbd0 con 0x7ff3900cf040 2024-03-23T07:57:54.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.412+0000 7ff38ea00640 1 -- 172.21.15.105:0/253440514 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880343f0 con 0x7ff3900cf040 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 -- 172.21.15.105:0/253440514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900cf040 msgr2=0x7ff3900b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 --2- 172.21.15.105:0/253440514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900cf040 0x7ff3900b1b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff388009e50 tx=0x7ff38802bed0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 -- 172.21.15.105:0/253440514 shutdown_connections 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 --2- 172.21.15.105:0/253440514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900cf040 0x7ff3900b1b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 --2- 172.21.15.105:0/253440514 >> v2:172.21.15.132:3300/0 conn(0x7ff390160d20 0x7ff390164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 -- 172.21.15.105:0/253440514 >> 172.21.15.105:0/253440514 conn(0x7ff39006a0b0 msgr2=0x7ff39006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 -- 172.21.15.105:0/253440514 shutdown_connections 2024-03-23T07:57:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.413+0000 7ff396000640 1 -- 172.21.15.105:0/253440514 wait complete. 2024-03-23T07:57:54.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.415+0000 7ff396000640 1 Processor -- start 2024-03-23T07:57:54.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.415+0000 7ff396000640 1 -- start start 2024-03-23T07:57:54.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff396000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900ca7b0 0x7ff39014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:54.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff396000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3900cf040 0x7ff3900b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:54.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff396000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff390160d20 0x7ff3901f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:54.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff396000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3901634e0 con 0x7ff3900cf040 2024-03-23T07:57:54.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff396000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900cd8e0 con 0x7ff390160d20 2024-03-23T07:57:54.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff396000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900cdbc0 con 0x7ff3900ca7b0 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff387e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3900cf040 0x7ff3900b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff387e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3900cf040 0x7ff3900b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49646/0 (socket says 172.21.15.105:49646) 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff387e00640 1 -- 172.21.15.105:0/1513144675 learned_addr learned my addr 172.21.15.105:0/1513144675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff38f400640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900ca7b0 0x7ff39014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff387e00640 1 -- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900ca7b0 msgr2=0x7ff39014e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.416+0000 7ff387e00640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900ca7b0 0x7ff39014e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff387e00640 1 -- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff390160d20 msgr2=0x7ff3901f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff38fe00640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff390160d20 0x7ff3901f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff387e00640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff390160d20 0x7ff3901f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff387e00640 1 -- 172.21.15.105:0/1513144675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff388009b30 con 0x7ff3900cf040 2024-03-23T07:57:54.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff387e00640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3900cf040 0x7ff3900b0ce0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c400 tx=0x7ff37800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:54.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff387400640 1 -- 172.21.15.105:0/1513144675 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780052d0 con 0x7ff3900cf040 2024-03-23T07:57:54.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff387400640 1 -- 172.21.15.105:0/1513144675 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff378005430 con 0x7ff3900cf040 2024-03-23T07:57:54.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff387400640 1 -- 172.21.15.105:0/1513144675 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378003830 con 0x7ff3900cf040 2024-03-23T07:57:54.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff390162e20 con 0x7ff3900cf040 2024-03-23T07:57:54.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.417+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3900b1220 con 0x7ff3900cf040 2024-03-23T07:57:54.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.418+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff390163620 con 0x7ff3900cf040 2024-03-23T07:57:54.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.419+0000 7ff387400640 1 -- 172.21.15.105:0/1513144675 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff378003990 con 0x7ff3900cf040 2024-03-23T07:57:54.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.421+0000 7ff387400640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36c06ac00 0x7ff36c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:54.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.421+0000 7ff387400640 1 -- 172.21.15.105:0/1513144675 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff378088ab0 con 0x7ff3900cf040 2024-03-23T07:57:54.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.421+0000 7ff38f400640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36c06ac00 0x7ff36c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:54.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.422+0000 7ff38f400640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36c06ac00 0x7ff36c06d0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff380009a10 tx=0x7ff380008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:54.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.425+0000 7ff387400640 1 -- 172.21.15.105:0/1513144675 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff378053ab0 con 0x7ff3900cf040 2024-03-23T07:57:54.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:54.640+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff39006def0 con 0x7ff3900cf040 2024-03-23T07:57:55.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.348+0000 7ff387400640 1 -- 172.21.15.105:0/1513144675 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff3780170b0 con 0x7ff3900cf040 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36c06ac00 msgr2=0x7ff36c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36c06ac00 0x7ff36c06d0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff380009a10 tx=0x7ff380008040 comp rx=0 tx=0).stop 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3900cf040 msgr2=0x7ff3900b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3900cf040 0x7ff3900b0ce0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c400 tx=0x7ff37800c8c0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 shutdown_connections 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff390160d20 0x7ff3901f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3900cf040 0x7ff3900b0ce0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff36c06ac00 0x7ff36c06d0c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 --2- 172.21.15.105:0/1513144675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3900ca7b0 0x7ff39014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 >> 172.21.15.105:0/1513144675 conn(0x7ff39006a0b0 msgr2=0x7ff3900b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:55.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 shutdown_connections 2024-03-23T07:57:55.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.352+0000 7ff396000640 1 -- 172.21.15.105:0/1513144675 wait complete. 2024-03-23T07:57:55.373 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T07:57:55.373 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T07:57:55.525 INFO:tasks.ceph.mds.a.smithi105.stderr:2024-03-23T07:57:55.525+0000 7f9579000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T07:57:55.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.586+0000 7f560a600640 1 Processor -- start 2024-03-23T07:57:55.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.586+0000 7f560a600640 1 -- start start 2024-03-23T07:57:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.587+0000 7f560a600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56040b84b0 0x7f56040d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.587+0000 7f560a600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56040d1280 con 0x7f56040cd740 2024-03-23T07:57:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.587+0000 7f560a600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56040d20c0 con 0x7f56040b5aa0 2024-03-23T07:57:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.587+0000 7f560a600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56040b8940 con 0x7f56040b84b0 2024-03-23T07:57:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5603400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f56040cd740 0x7f56040cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:34036/0 (socket says 172.21.15.105:34036) 2024-03-23T07:57:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5603400640 1 -- 172.21.15.105:0/1537895088 learned_addr learned my addr 172.21.15.105:0/1537895088 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5602a00640 1 --2- 172.21.15.105:0/1537895088 >> v2:172.21.15.105:3300/0 conn(0x7f56040b84b0 0x7f56040d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5602000640 1 -- 172.21.15.105:0/1537895088 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349516985 0 0) 0x7f56040d1280 con 0x7f56040cd740 2024-03-23T07:57:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5602000640 1 -- 172.21.15.105:0/1537895088 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e8003520 con 0x7f56040cd740 2024-03-23T07:57:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5602a00640 1 -- 172.21.15.105:0/1537895088 >> v1:172.21.15.132:6789/0 conn(0x7f56040b5aa0 legacy=0x7f56040b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5602a00640 1 -- 172.21.15.105:0/1537895088 >> v1:172.21.15.105:6789/0 conn(0x7f56040cd740 legacy=0x7f56040cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.588+0000 7f5602a00640 1 -- 172.21.15.105:0/1537895088 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56040d14b0 con 0x7f56040b84b0 2024-03-23T07:57:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.590+0000 7f5602a00640 1 --2- 172.21.15.105:0/1537895088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f56040d0d40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f55f00097a0 tx=0x7f55f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794f1a4f0b55cc83 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.590+0000 7f5602000640 1 -- 172.21.15.105:0/1537895088 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0030610 con 0x7f56040b84b0 2024-03-23T07:57:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.590+0000 7f5602000640 1 -- 172.21.15.105:0/1537895088 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0030770 con 0x7f56040b84b0 2024-03-23T07:57:55.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.590+0000 7f5602000640 1 -- 172.21.15.105:0/1537895088 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0030b00 con 0x7f56040b84b0 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.591+0000 7f560a600640 1 -- 172.21.15.105:0/1537895088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 msgr2=0x7f56040d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.591+0000 7f560a600640 1 --2- 172.21.15.105:0/1537895088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f56040d0d40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f55f00097a0 tx=0x7f55f002b960 comp rx=0 tx=0).stop 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.591+0000 7f560a600640 1 -- 172.21.15.105:0/1537895088 shutdown_connections 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.591+0000 7f560a600640 1 --2- 172.21.15.105:0/1537895088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f56040d0d40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.591+0000 7f560a600640 1 -- 172.21.15.105:0/1537895088 >> 172.21.15.105:0/1537895088 conn(0x7f56040cf5d0 msgr2=0x7f56040cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.591+0000 7f560a600640 1 -- 172.21.15.105:0/1537895088 shutdown_connections 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.591+0000 7f560a600640 1 -- 172.21.15.105:0/1537895088 wait complete. 2024-03-23T07:57:55.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.593+0000 7f560a600640 1 Processor -- start 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.593+0000 7f560a600640 1 -- start start 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f560a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56040b5aa0 0x7f56040a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f560a600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f560414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f560a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56040cd740 0x7f56040cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f560a600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5604150020 con 0x7f56040b84b0 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f560a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56040cc3b0 con 0x7f56040cd740 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f560a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56040a9900 con 0x7f56040b5aa0 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f560414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f560414fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49664/0 (socket says 172.21.15.105:49664) 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 -- 172.21.15.105:0/2440226785 learned_addr learned my addr 172.21.15.105:0/2440226785 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5603400640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56040b5aa0 0x7f56040a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 -- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56040b5aa0 msgr2=0x7f56040a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56040b5aa0 0x7f56040a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 -- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56040cd740 msgr2=0x7f56040cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56040cd740 0x7f56040cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.594+0000 7f5602a00640 1 -- 172.21.15.105:0/2440226785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0009400 con 0x7f56040b84b0 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.595+0000 7f5603400640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56040b5aa0 0x7f56040a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.595+0000 7f5602a00640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f560414fae0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f55f002bf70 tx=0x7f55f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.595+0000 7f55fb400640 1 -- 172.21.15.105:0/2440226785 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f003e040 con 0x7f56040b84b0 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.595+0000 7f55fb400640 1 -- 172.21.15.105:0/2440226785 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0039cb0 con 0x7f56040b84b0 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.595+0000 7f55fb400640 1 -- 172.21.15.105:0/2440226785 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0044630 con 0x7f56040b84b0 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.595+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56040a9be0 con 0x7f56040b84b0 2024-03-23T07:57:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.595+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56040aa1a0 con 0x7f56040b84b0 2024-03-23T07:57:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.596+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5604073390 con 0x7f56040b84b0 2024-03-23T07:57:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.598+0000 7f55fb400640 1 -- 172.21.15.105:0/2440226785 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f55f0043390 con 0x7f56040b84b0 2024-03-23T07:57:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.599+0000 7f55fb400640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f55e406ac30 0x7f55e406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.599+0000 7f55fb400640 1 -- 172.21.15.105:0/2440226785 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f55f00ab640 con 0x7f56040b84b0 2024-03-23T07:57:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.599+0000 7f5603400640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f55e406ac30 0x7f55e406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.600+0000 7f5603400640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f55e406ac30 0x7f55e406d0f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55f40011e0 tx=0x7f55f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:55.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.604+0000 7f55fb400640 1 -- 172.21.15.105:0/2440226785 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f55f00766d0 con 0x7f56040b84b0 2024-03-23T07:57:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:55.837+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f56040b2050 con 0x7f56040b84b0 2024-03-23T07:57:56.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.352+0000 7f55fb400640 1 -- 172.21.15.105:0/2440226785 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55f0079cf0 con 0x7f56040b84b0 2024-03-23T07:57:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.355+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f55e406ac30 msgr2=0x7f55e406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.355+0000 7f560a600640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f55e406ac30 0x7f55e406d0f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55f40011e0 tx=0x7f55f4009040 comp rx=0 tx=0).stop 2024-03-23T07:57:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.355+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 msgr2=0x7f560414fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.355+0000 7f560a600640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f560414fae0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f55f002bf70 tx=0x7f55f0004730 comp rx=0 tx=0).stop 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 shutdown_connections 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56040cd740 0x7f56040cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56040b84b0 0x7f560414fae0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f55e406ac30 0x7f55e406d0f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 --2- 172.21.15.105:0/2440226785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56040b5aa0 0x7f56040a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 >> 172.21.15.105:0/2440226785 conn(0x7f56040cf5d0 msgr2=0x7f560415dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 shutdown_connections 2024-03-23T07:57:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.356+0000 7f560a600640 1 -- 172.21.15.105:0/2440226785 wait complete. 2024-03-23T07:57:56.373 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T07:57:56.373 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:57:56.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.580+0000 7f9548400640 1 Processor -- start 2024-03-23T07:57:56.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.580+0000 7f9548400640 1 -- start start 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9548400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95400b5a30 0x7f95400b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9548400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95400b4350 0x7f95400b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9548400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95400d1830 con 0x7f954015cda0 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9548400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95400b4d00 con 0x7f95400b5a30 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9548400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95400b1f20 con 0x7f95400b4350 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9545000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95400b5a30 0x7f95400b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9545000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95400b5a30 0x7f95400b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49676/0 (socket says 172.21.15.105:49676) 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9545000640 1 -- 172.21.15.105:0/418767252 learned_addr learned my addr 172.21.15.105:0/418767252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:56.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9545a00640 1 --1- 172.21.15.105:0/418767252 >> v1:172.21.15.105:6789/0 conn(0x7f954015cda0 0x7f95400cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:34042/0 (socket says 172.21.15.105:34042) 2024-03-23T07:57:56.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.582+0000 7f9545000640 1 -- 172.21.15.105:0/418767252 >> v2:172.21.15.132:3300/0 conn(0x7f95400b4350 msgr2=0x7f95400b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:56.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.583+0000 7f9545000640 1 --2- 172.21.15.105:0/418767252 >> v2:172.21.15.132:3300/0 conn(0x7f95400b4350 0x7f95400b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.583+0000 7f9545000640 1 -- 172.21.15.105:0/418767252 >> v1:172.21.15.105:6789/0 conn(0x7f954015cda0 legacy=0x7f95400cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.583+0000 7f9545000640 1 -- 172.21.15.105:0/418767252 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400d0c20 con 0x7f95400b5a30 2024-03-23T07:57:56.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.584+0000 7f9545000640 1 --2- 172.21.15.105:0/418767252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95400b5a30 0x7f95400b5e90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f95400b8460 tx=0x7f952c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2113d481b57a5729 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.584+0000 7f953fe00640 1 -- 172.21.15.105:0/418767252 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c030870 con 0x7f95400b5a30 2024-03-23T07:57:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.584+0000 7f953fe00640 1 -- 172.21.15.105:0/418767252 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c0309d0 con 0x7f95400b5a30 2024-03-23T07:57:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.584+0000 7f953fe00640 1 -- 172.21.15.105:0/418767252 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c030d60 con 0x7f95400b5a30 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 -- 172.21.15.105:0/418767252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95400b5a30 msgr2=0x7f95400b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 --2- 172.21.15.105:0/418767252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95400b5a30 0x7f95400b5e90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f95400b8460 tx=0x7f952c02b9b0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 -- 172.21.15.105:0/418767252 shutdown_connections 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 --2- 172.21.15.105:0/418767252 >> v2:172.21.15.132:3300/0 conn(0x7f95400b4350 0x7f95400b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 --2- 172.21.15.105:0/418767252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95400b5a30 0x7f95400b5e90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 -- 172.21.15.105:0/418767252 >> 172.21.15.105:0/418767252 conn(0x7f95400cf350 msgr2=0x7f95400cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 -- 172.21.15.105:0/418767252 shutdown_connections 2024-03-23T07:57:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.585+0000 7f9548400640 1 -- 172.21.15.105:0/418767252 wait complete. 2024-03-23T07:57:56.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.587+0000 7f9548400640 1 Processor -- start 2024-03-23T07:57:56.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.587+0000 7f9548400640 1 -- start start 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.588+0000 7f9548400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95400b4350 0x7f95401f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.588+0000 7f9548400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95400b5a30 0x7f9540150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.588+0000 7f9548400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f954015cda0 0x7f95400d2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.588+0000 7f9548400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401f34e0 con 0x7f954015cda0 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.588+0000 7f9548400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954015dc50 con 0x7f95400b5a30 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.588+0000 7f9548400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954015d320 con 0x7f95400b4350 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.588+0000 7f9546400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f954015cda0 0x7f95400d2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f954015cda0 0x7f95400d2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49682/0 (socket says 172.21.15.105:49682) 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 -- 172.21.15.105:0/3900121188 learned_addr learned my addr 172.21.15.105:0/3900121188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9545a00640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95400b4350 0x7f95401f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9545000640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95400b5a30 0x7f9540150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:56.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 -- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95400b4350 msgr2=0x7f95401f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95400b4350 0x7f95401f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 -- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95400b5a30 msgr2=0x7f9540150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95400b5a30 0x7f9540150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 -- 172.21.15.105:0/3900121188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0094d0 con 0x7f954015cda0 2024-03-23T07:57:56.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.589+0000 7f9546400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f954015cda0 0x7f95400d2d20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f953800b9a0 tx=0x7f953800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:56.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.590+0000 7f953d600640 1 -- 172.21.15.105:0/3900121188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953800cba0 con 0x7f954015cda0 2024-03-23T07:57:56.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.590+0000 7f953d600640 1 -- 172.21.15.105:0/3900121188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953800cd20 con 0x7f954015cda0 2024-03-23T07:57:56.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.590+0000 7f953d600640 1 -- 172.21.15.105:0/3900121188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538014680 con 0x7f954015cda0 2024-03-23T07:57:56.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.590+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954015d600 con 0x7f954015cda0 2024-03-23T07:57:56.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.590+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9540151220 con 0x7f954015cda0 2024-03-23T07:57:56.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.591+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9540066500 con 0x7f954015cda0 2024-03-23T07:57:56.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.592+0000 7f953d600640 1 -- 172.21.15.105:0/3900121188 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f953801d3b0 con 0x7f954015cda0 2024-03-23T07:57:56.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.593+0000 7f953d600640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952006ac00 0x7f952006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:56.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.593+0000 7f953d600640 1 -- 172.21.15.105:0/3900121188 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9538089390 con 0x7f954015cda0 2024-03-23T07:57:56.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.593+0000 7f9545a00640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952006ac00 0x7f952006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:56.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.594+0000 7f9545a00640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952006ac00 0x7f952006d0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9534000f60 tx=0x7f9534007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:56.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.598+0000 7f953d600640 1 -- 172.21.15.105:0/3900121188 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f95380544c0 con 0x7f954015cda0 2024-03-23T07:57:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.832+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9540073390 con 0x7f954015cda0 2024-03-23T07:57:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.833+0000 7f953d600640 1 -- 172.21.15.105:0/3900121188 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f9538057ae0 con 0x7f954015cda0 2024-03-23T07:57:56.832 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:56.833 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:57:56.345100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/3723879365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3723879365},{"type":"v1","addr":"172.21.15.105:6837","nonce":3723879365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:57:56.833 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 16 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952006ac00 msgr2=0x7f952006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952006ac00 0x7f952006d0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9534000f60 tx=0x7f9534007c80 comp rx=0 tx=0).stop 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f954015cda0 msgr2=0x7f95400d2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f954015cda0 0x7f95400d2d20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f953800b9a0 tx=0x7f953800be60 comp rx=0 tx=0).stop 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 shutdown_connections 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f954015cda0 0x7f95400d2d20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95400b5a30 0x7f9540150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f952006ac00 0x7f952006d0c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 --2- 172.21.15.105:0/3900121188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95400b4350 0x7f95401f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 >> 172.21.15.105:0/3900121188 conn(0x7f95400cf350 msgr2=0x7f95400b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 shutdown_connections 2024-03-23T07:57:56.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:56.836+0000 7f9548400640 1 -- 172.21.15.105:0/3900121188 wait complete. 2024-03-23T07:57:56.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:57:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.060+0000 7f1487200640 1 Processor -- start 2024-03-23T07:57:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.061+0000 7f1487200640 1 -- start start 2024-03-23T07:57:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.062+0000 7f1487200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1480142f90 0x7f14800cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.062+0000 7f1487200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14800cc750 0x7f14800b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.062+0000 7f1487200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14800cb7f0 con 0x7f14800b3190 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.062+0000 7f1487200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14800b25b0 con 0x7f1480142f90 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.062+0000 7f1487200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14800ccb90 con 0x7f14800cc750 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f147fe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1480142f90 0x7f14800cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f147fe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1480142f90 0x7f14800cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43970/0 (socket says 172.21.15.105:43970) 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f147fe00640 1 -- 172.21.15.105:0/534367561 learned_addr learned my addr 172.21.15.105:0/534367561 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f147fe00640 1 -- 172.21.15.105:0/534367561 >> v2:172.21.15.132:3300/0 conn(0x7f14800cc750 msgr2=0x7f14800b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f1477400640 1 --2- 172.21.15.105:0/534367561 >> v2:172.21.15.132:3300/0 conn(0x7f14800cc750 0x7f14800b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f147fe00640 1 --2- 172.21.15.105:0/534367561 >> v2:172.21.15.132:3300/0 conn(0x7f14800cc750 0x7f14800b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f147fe00640 1 -- 172.21.15.105:0/534367561 >> v1:172.21.15.105:6789/0 conn(0x7f14800b3190 legacy=0x7f1480160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.063+0000 7f147fe00640 1 -- 172.21.15.105:0/534367561 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480143370 con 0x7f1480142f90 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.064+0000 7f147fe00640 1 --2- 172.21.15.105:0/534367561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480142f90 0x7f14800cc210 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f14800cdff0 tx=0x7f147002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a314803ca1b889 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.064+0000 7f147f400640 1 -- 172.21.15.105:0/534367561 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470030860 con 0x7f1480142f90 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f147f400640 1 -- 172.21.15.105:0/534367561 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14700309c0 con 0x7f1480142f90 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f147f400640 1 -- 172.21.15.105:0/534367561 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470030d50 con 0x7f1480142f90 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 -- 172.21.15.105:0/534367561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480142f90 msgr2=0x7f14800cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 --2- 172.21.15.105:0/534367561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480142f90 0x7f14800cc210 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f14800cdff0 tx=0x7f147002b9a0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 -- 172.21.15.105:0/534367561 shutdown_connections 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 --2- 172.21.15.105:0/534367561 >> v2:172.21.15.132:3300/0 conn(0x7f14800cc750 0x7f14800b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 --2- 172.21.15.105:0/534367561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480142f90 0x7f14800cc210 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 -- 172.21.15.105:0/534367561 >> 172.21.15.105:0/534367561 conn(0x7f1480065e20 msgr2=0x7f1480069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 -- 172.21.15.105:0/534367561 shutdown_connections 2024-03-23T07:57:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.065+0000 7f1487200640 1 -- 172.21.15.105:0/534367561 wait complete. 2024-03-23T07:57:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.067+0000 7f1487200640 1 Processor -- start 2024-03-23T07:57:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.067+0000 7f1487200640 1 -- start start 2024-03-23T07:57:57.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1487200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14800b3190 0x7f1480144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1487200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14800cc750 0x7f1480141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1487200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1480142f90 0x7f1480159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1487200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480144c70 con 0x7f14800cc750 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1487200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801f6810 con 0x7f1480142f90 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1487200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148015cec0 con 0x7f14800b3190 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1477400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14800cc750 0x7f1480141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1477400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14800cc750 0x7f1480141ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43984/0 (socket says 172.21.15.105:43984) 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.068+0000 7f1477400640 1 -- 172.21.15.105:0/3377589417 learned_addr learned my addr 172.21.15.105:0/3377589417 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f147fe00640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14800b3190 0x7f1480144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1485200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1480142f90 0x7f1480159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1477400640 1 -- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14800b3190 msgr2=0x7f1480144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1477400640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14800b3190 0x7f1480144730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1477400640 1 -- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1480142f90 msgr2=0x7f1480159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1477400640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1480142f90 0x7f1480159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1477400640 1 -- 172.21.15.105:0/3377589417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14700094d0 con 0x7f14800cc750 2024-03-23T07:57:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1485200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1480142f90 0x7f1480159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1477400640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14800cc750 0x7f1480141ca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f146800c7f0 tx=0x7f146800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f147cc00640 1 -- 172.21.15.105:0/3377589417 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468004220 con 0x7f14800cc750 2024-03-23T07:57:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f147cc00640 1 -- 172.21.15.105:0/3377589417 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14680043a0 con 0x7f14800cc750 2024-03-23T07:57:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.069+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148015d1a0 con 0x7f14800cc750 2024-03-23T07:57:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.070+0000 7f147cc00640 1 -- 172.21.15.105:0/3377589417 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468007860 con 0x7f14800cc750 2024-03-23T07:57:57.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.070+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14801421e0 con 0x7f14800cc750 2024-03-23T07:57:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.071+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148015d6c0 con 0x7f14800cc750 2024-03-23T07:57:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.072+0000 7f147cc00640 1 -- 172.21.15.105:0/3377589417 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1468007bd0 con 0x7f14800cc750 2024-03-23T07:57:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.072+0000 7f147cc00640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f145c06ac00 0x7f145c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.073+0000 7f147cc00640 1 -- 172.21.15.105:0/3377589417 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1468089400 con 0x7f14800cc750 2024-03-23T07:57:57.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.073+0000 7f147fe00640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f145c06ac00 0x7f145c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.073+0000 7f147fe00640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f145c06ac00 0x7f145c06d0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1470036000 tx=0x7f147003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.077+0000 7f147cc00640 1 -- 172.21.15.105:0/3377589417 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f148015d6c0 con 0x7f14800cc750 2024-03-23T07:57:57.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.271+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f148015d6c0 con 0x7f14800cc750 2024-03-23T07:57:57.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.274+0000 7f147cc00640 1 -- 172.21.15.105:0/3377589417 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16354 (secure 0 0 0) 0x7f148015d6c0 con 0x7f14800cc750 2024-03-23T07:57:57.274 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:57.274 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":41,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:57:54.329653+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:57:51.204124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-23T07:57:52.349446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f145c06ac00 msgr2=0x7f145c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f145c06ac00 0x7f145c06d0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1470036000 tx=0x7f147003b040 comp rx=0 tx=0).stop 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14800cc750 msgr2=0x7f1480141ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14800cc750 0x7f1480141ca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f146800c7f0 tx=0x7f146800ccb0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 shutdown_connections 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1480142f90 0x7f1480159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14800cc750 0x7f1480141ca0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f145c06ac00 0x7f145c06d0c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 --2- 172.21.15.105:0/3377589417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14800b3190 0x7f1480144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 >> 172.21.15.105:0/3377589417 conn(0x7f1480065e20 msgr2=0x7f1480070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 shutdown_connections 2024-03-23T07:57:57.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.277+0000 7f1487200640 1 -- 172.21.15.105:0/3377589417 wait complete. 2024-03-23T07:57:57.296 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T07:57:57.296 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:57:57.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.505+0000 7f100c400640 1 Processor -- start 2024-03-23T07:57:57.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.506+0000 7f100c400640 1 -- start start 2024-03-23T07:57:57.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.507+0000 7f100c400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f10040b5a30 0x7f10040b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.507+0000 7f100c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 0x7f10040b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.507+0000 7f100c400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040d1830 con 0x7f100415cda0 2024-03-23T07:57:57.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.507+0000 7f100c400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040b4d20 con 0x7f10040b5a30 2024-03-23T07:57:57.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.507+0000 7f100c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040b1f20 con 0x7f10040b4350 2024-03-23T07:57:57.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.507+0000 7f1009a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f100415cda0 0x7f10040cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42368/0 (socket says 172.21.15.105:42368) 2024-03-23T07:57:57.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.507+0000 7f1009a00640 1 -- 172.21.15.105:0/2385653454 learned_addr learned my addr 172.21.15.105:0/2385653454 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1009000640 1 --2- 172.21.15.105:0/2385653454 >> v2:172.21.15.132:3300/0 conn(0x7f10040b5a30 0x7f10040b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f100a400640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 0x7f10040b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857510987 0 0) 0x7f10040d1830 con 0x7f100415cda0 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fec003520 con 0x7f100415cda0 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1593774230 0 0) 0x7f0fec003520 con 0x7f100415cda0 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 msgr2=0x7f10040b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 0x7f10040b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 >> v2:172.21.15.132:3300/0 conn(0x7f10040b5a30 msgr2=0x7f10040b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 --2- 172.21.15.105:0/2385653454 >> v2:172.21.15.132:3300/0 conn(0x7f10040b5a30 0x7f10040b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040d0c20 con 0x7f100415cda0 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f0ff8001670 con 0x7f100415cda0 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 >> v1:172.21.15.105:6789/0 conn(0x7f100415cda0 legacy=0x7f10040cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 0x7f0fec003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec003ef0 0x7f0fec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec004d00 0x7f0fec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040b4d20 con 0x7f0fec004d00 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040b1f20 con 0x7f0fec003ef0 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.508+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec00b9d0 con 0x7f10040b4350 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f100a400640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec004d00 0x7f0fec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f1009a00640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 0x7f0fec003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f1009000640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec003ef0 0x7f0fec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f100a400640 1 -- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 msgr2=0x7f0fec003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f100a400640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040b4350 0x7f0fec003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f100a400640 1 -- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec003ef0 msgr2=0x7f0fec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f100a400640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec003ef0 0x7f0fec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f100a400640 1 -- 172.21.15.105:0/2385653454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec003520 con 0x7f0fec004d00 2024-03-23T07:57:57.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.509+0000 7f1009000640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec003ef0 0x7f0fec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.510+0000 7f100a400640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec004d00 0x7f0fec005180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc007360 tx=0x7f0fec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db70b7fa1cbe113 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc02ccd0 con 0x7f0fec004d00 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc02ce30 con 0x7f0fec004d00 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f1003e00640 1 -- 172.21.15.105:0/2385653454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc034660 con 0x7f0fec004d00 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f100c400640 1 -- 172.21.15.105:0/2385653454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec004d00 msgr2=0x7f0fec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f100c400640 1 --2- 172.21.15.105:0/2385653454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec004d00 0x7f0fec005180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc007360 tx=0x7f0fec0057f0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f100a400640 1 -- 172.21.15.105:0/2385653454 reap_dead start 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f100c400640 1 -- 172.21.15.105:0/2385653454 shutdown_connections 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f100c400640 1 -- 172.21.15.105:0/2385653454 >> 172.21.15.105:0/2385653454 conn(0x7f10040cf350 msgr2=0x7f10040cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f100c400640 1 -- 172.21.15.105:0/2385653454 shutdown_connections 2024-03-23T07:57:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.511+0000 7f100c400640 1 -- 172.21.15.105:0/2385653454 wait complete. 2024-03-23T07:57:57.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.513+0000 7f100c400640 1 Processor -- start 2024-03-23T07:57:57.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.513+0000 7f100c400640 1 -- start start 2024-03-23T07:57:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.514+0000 7f100c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec004d00 0x7f10041f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.514+0000 7f100c400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1004153c50 0x7f100415d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.514+0000 7f100c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040c8820 0x7f10040c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.514+0000 7f100c400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040c91e0 con 0x7f1004153c50 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.514+0000 7f100c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004154c60 con 0x7f0fec004d00 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.514+0000 7f100c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040ca230 con 0x7f10040c8820 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1004153c50 0x7f100415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1004153c50 0x7f100415d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44004/0 (socket says 172.21.15.105:44004) 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 -- 172.21.15.105:0/1099678726 learned_addr learned my addr 172.21.15.105:0/1099678726 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009a00640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec004d00 0x7f10041f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f100a400640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040c8820 0x7f10040c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 -- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040c8820 msgr2=0x7f10040c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040c8820 0x7f10040c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 -- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec004d00 msgr2=0x7f10041f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec004d00 0x7f10041f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 -- 172.21.15.105:0/1099678726 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10041543c0 con 0x7f1004153c50 2024-03-23T07:57:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.515+0000 7f1009000640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1004153c50 0x7f100415d480 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4011600 tx=0x7f0ff4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.516+0000 7f1001600640 1 -- 172.21.15.105:0/1099678726 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff400f920 con 0x7f1004153c50 2024-03-23T07:57:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.516+0000 7f1001600640 1 -- 172.21.15.105:0/1099678726 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff400fa80 con 0x7f1004153c50 2024-03-23T07:57:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.516+0000 7f1001600640 1 -- 172.21.15.105:0/1099678726 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff400fe10 con 0x7f1004153c50 2024-03-23T07:57:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.516+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10041546a0 con 0x7f1004153c50 2024-03-23T07:57:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.516+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10041540d0 con 0x7f1004153c50 2024-03-23T07:57:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.517+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1004073390 con 0x7f1004153c50 2024-03-23T07:57:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.518+0000 7f1001600640 1 -- 172.21.15.105:0/1099678726 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0ff4015460 con 0x7f1004153c50 2024-03-23T07:57:57.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.519+0000 7f1001600640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0fe806ac30 0x7f0fe806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:57.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.519+0000 7f1001600640 1 -- 172.21.15.105:0/1099678726 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0ff4089cb0 con 0x7f1004153c50 2024-03-23T07:57:57.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.519+0000 7f1009a00640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0fe806ac30 0x7f0fe806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:57.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.520+0000 7f1009a00640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0fe806ac30 0x7f0fe806d0f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f10040c9de0 tx=0x7f0ff8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:57.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.524+0000 7f1001600640 1 -- 172.21.15.105:0/1099678726 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0ff4054d40 con 0x7f1004153c50 2024-03-23T07:57:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.712+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10040d14c0 con 0x7f0fe806ac30 2024-03-23T07:57:57.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.735+0000 7f1001600640 1 -- 172.21.15.105:0/1099678726 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274850 (secure 0 0 0) 0x7f0ff8010930 con 0x7f0fe806ac30 2024-03-23T07:57:57.739 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:57.741 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:57:57.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0fe806ac30 msgr2=0x7f0fe806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0fe806ac30 0x7f0fe806d0f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f10040c9de0 tx=0x7f0ff8009040 comp rx=0 tx=0).stop 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1004153c50 msgr2=0x7f100415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1004153c50 0x7f100415d480 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4011600 tx=0x7f0ff4011ac0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 shutdown_connections 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0fe806ac30 0x7f0fe806d0f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10040c8820 0x7f10040c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1004153c50 0x7f100415d480 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 --2- 172.21.15.105:0/1099678726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fec004d00 0x7f10041f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 >> 172.21.15.105:0/1099678726 conn(0x7f10040cf350 msgr2=0x7f10040cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 shutdown_connections 2024-03-23T07:57:57.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:57.742+0000 7f100c400640 1 -- 172.21.15.105:0/1099678726 wait complete. 2024-03-23T07:57:57.762 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-03-23T07:57:56.074259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":44,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":220668,"kb_used_data":4444,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754052,"statfs":{"total":773094113280,"available":772868149248,"internally_reserved":0,"allocated":4550656,"data_stored":2576922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.225390"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352583+0000","last_change":"2024-03-23T07:57:53.374646+0000","last_active":"2024-03-23T07:57:54.352583+0000","last_peered":"2024-03-23T07:57:54.352583+0000","last_clean":"2024-03-23T07:57:54.352583+0000","last_became_active":"2024-03-23T07:57:53.374380+0000","last_became_peered":"2024-03-23T07:57:53.374380+0000","last_unstale":"2024-03-23T07:57:54.352583+0000","last_undegraded":"2024-03-23T07:57:54.352583+0000","last_fullsized":"2024-03-23T07:57:54.352583+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:05.821253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.854401+0000","last_change":"2024-03-23T07:57:51.854401+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.854401+0000","last_undegraded":"2024-03-23T07:57:51.854401+0000","last_fullsized":"2024-03-23T07:57:51.854401+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:52.674138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.854167+0000","last_change":"2024-03-23T07:57:51.854167+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.854167+0000","last_undegraded":"2024-03-23T07:57:51.854167+0000","last_fullsized":"2024-03-23T07:57:51.854167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:04:12.427237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.854467+0000","last_change":"2024-03-23T07:57:51.854467+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.854467+0000","last_undegraded":"2024-03-23T07:57:51.854467+0000","last_fullsized":"2024-03-23T07:57:51.854467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:26:33.909802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.858729+0000","last_change":"2024-03-23T07:57:51.858729+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.858729+0000","last_undegraded":"2024-03-23T07:57:51.858729+0000","last_fullsized":"2024-03-23T07:57:51.858729+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:15:33.069065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.858218+0000","last_change":"2024-03-23T07:57:51.858218+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.858218+0000","last_undegraded":"2024-03-23T07:57:51.858218+0000","last_fullsized":"2024-03-23T07:57:51.858218+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:18:59.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.863257+0000","last_change":"2024-03-23T07:57:51.863257+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.863257+0000","last_undegraded":"2024-03-23T07:57:51.863257+0000","last_fullsized":"2024-03-23T07:57:51.863257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:49:40.406677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.862309+0000","last_change":"2024-03-23T07:57:51.862309+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.862309+0000","last_undegraded":"2024-03-23T07:57:51.862309+0000","last_fullsized":"2024-03-23T07:57:51.862309+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:10.577534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.863651+0000","last_change":"2024-03-23T07:57:51.863651+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.863651+0000","last_undegraded":"2024-03-23T07:57:51.863651+0000","last_fullsized":"2024-03-23T07:57:51.863651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:24:14.842764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.855141+0000","last_change":"2024-03-23T07:57:51.855141+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.855141+0000","last_undegraded":"2024-03-23T07:57:51.855141+0000","last_fullsized":"2024-03-23T07:57:51.855141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:42.255496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.858532+0000","last_change":"2024-03-23T07:57:51.858532+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.858532+0000","last_undegraded":"2024-03-23T07:57:51.858532+0000","last_fullsized":"2024-03-23T07:57:51.858532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:51:15.459650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.862020+0000","last_change":"2024-03-23T07:57:51.862020+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.862020+0000","last_undegraded":"2024-03-23T07:57:51.862020+0000","last_fullsized":"2024-03-23T07:57:51.862020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:42.132614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352855+0000","last_change":"2024-03-23T07:57:53.376624+0000","last_active":"2024-03-23T07:57:54.352855+0000","last_peered":"2024-03-23T07:57:54.352855+0000","last_clean":"2024-03-23T07:57:54.352855+0000","last_became_active":"2024-03-23T07:57:53.376361+0000","last_became_peered":"2024-03-23T07:57:53.376361+0000","last_unstale":"2024-03-23T07:57:54.352855+0000","last_undegraded":"2024-03-23T07:57:54.352855+0000","last_fullsized":"2024-03-23T07:57:54.352855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:07:12.470373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352347+0000","last_change":"2024-03-23T07:57:52.888444+0000","last_active":"2024-03-23T07:57:54.352347+0000","last_peered":"2024-03-23T07:57:54.352347+0000","last_clean":"2024-03-23T07:57:54.352347+0000","last_became_active":"2024-03-23T07:57:52.888147+0000","last_became_peered":"2024-03-23T07:57:52.888147+0000","last_unstale":"2024-03-23T07:57:54.352347+0000","last_undegraded":"2024-03-23T07:57:54.352347+0000","last_fullsized":"2024-03-23T07:57:54.352347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:01.159974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352081+0000","last_change":"2024-03-23T07:57:52.901586+0000","last_active":"2024-03-23T07:57:54.352081+0000","last_peered":"2024-03-23T07:57:54.352081+0000","last_clean":"2024-03-23T07:57:54.352081+0000","last_became_active":"2024-03-23T07:57:52.901313+0000","last_became_peered":"2024-03-23T07:57:52.901313+0000","last_unstale":"2024-03-23T07:57:54.352081+0000","last_undegraded":"2024-03-23T07:57:54.352081+0000","last_fullsized":"2024-03-23T07:57:54.352081+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:49:25.759750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.858373+0000","last_change":"2024-03-23T07:57:51.858373+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.858373+0000","last_undegraded":"2024-03-23T07:57:51.858373+0000","last_fullsized":"2024-03-23T07:57:51.858373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:10.882850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.862094+0000","last_change":"2024-03-23T07:57:51.862094+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.862094+0000","last_undegraded":"2024-03-23T07:57:51.862094+0000","last_fullsized":"2024-03-23T07:57:51.862094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:17:52.525976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352607+0000","last_change":"2024-03-23T07:57:53.364875+0000","last_active":"2024-03-23T07:57:54.352607+0000","last_peered":"2024-03-23T07:57:54.352607+0000","last_clean":"2024-03-23T07:57:54.352607+0000","last_became_active":"2024-03-23T07:57:53.363916+0000","last_became_peered":"2024-03-23T07:57:53.363916+0000","last_unstale":"2024-03-23T07:57:54.352607+0000","last_undegraded":"2024-03-23T07:57:54.352607+0000","last_fullsized":"2024-03-23T07:57:54.352607+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:04.554973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.861623+0000","last_change":"2024-03-23T07:57:51.861623+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.861623+0000","last_undegraded":"2024-03-23T07:57:51.861623+0000","last_fullsized":"2024-03-23T07:57:51.861623+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:19.123270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.863157+0000","last_change":"2024-03-23T07:57:51.863157+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.863157+0000","last_undegraded":"2024-03-23T07:57:51.863157+0000","last_fullsized":"2024-03-23T07:57:51.863157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:39:26.312255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:51.833420+0000","last_change":"2024-03-23T07:57:51.833420+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.833420+0000","last_undegraded":"2024-03-23T07:57:51.833420+0000","last_fullsized":"2024-03-23T07:57:51.833420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T07:57:52.848837+0000","last_change":"2024-03-23T07:57:52.848837+0000","last_active":"2024-03-23T07:57:52.848837+0000","last_peered":"2024-03-23T07:57:52.848837+0000","last_clean":"2024-03-23T07:57:52.848837+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:52.848837+0000","last_undegraded":"2024-03-23T07:57:52.848837+0000","last_fullsized":"2024-03-23T07:57:52.848837+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-23T07:57:51.861733+0000","last_change":"2024-03-23T07:57:51.861733+0000","last_active":"2024-03-23T07:57:51.833420+0000","last_peered":"2024-03-23T07:57:51.833420+0000","last_clean":"2024-03-23T07:57:51.833420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T07:57:51.861733+0000","last_undegraded":"2024-03-23T07:57:51.861733+0000","last_fullsized":"2024-03-23T07:57:51.861733+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:29:39.714121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-23T07:57:46.289262+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:57:46.289262+0000","last_peered":"2024-03-23T07:57:46.289262+0000","last_clean":"2024-03-23T07:57:46.289262+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:57:46.289262+0000","last_undegraded":"2024-03-23T07:57:46.289262+0000","last_fullsized":"2024-03-23T07:57:46.289262+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":36,"acting":36,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476760,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":177342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.376},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.55400000000000005}]},{"osd":1,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.40400000000000003},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.60399999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.0149999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.077}]},{"osd":6,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.876},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":3.6899999999999999,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":0.78800000000000003}]},{"osd":7,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.2210000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476760,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":171205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.39800000000000002}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.48999999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.873}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.3420000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":0.76100000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8410000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.573,"15min":1.573},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.127}]}]},{"osd":1,"up_from":16,"seq":68719476760,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":761573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.312},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.1699999999999999,"5min":3.1699999999999999,"15min":3.1699999999999999},"last":0.35499999999999998}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":5.8109999999999999,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.95899999999999996}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.052},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.73699999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.121}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":1.095}]}]},{"osd":0,"up_from":16,"seq":68719476762,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":413696,"data_stored":171205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.85299999999999998}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.3080000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.286},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.0469999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":1.5109999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.399},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.129}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.363},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.47}]}]},{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":413696,"data_stored":171205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.101}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.745},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.0409999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.98299999999999998}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.131},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.3240000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":1.006,"5min":1.006,"15min":1.006},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.258}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.548},{"interface":"front","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.444}]}]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":413696,"data_stored":171205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.0249999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.746},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.3720000000000001}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.3520000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.2749999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.468}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.319}]}]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":413696,"data_stored":171205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.048},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.1970000000000001}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.1579999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.999}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.361},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.244}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4590000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.3260000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.60499999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.80000000000000004}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":1.29,"5min":1.29,"15min":1.29},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.94399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:57:57.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:57:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.017+0000 7f1573e00640 1 Processor -- start 2024-03-23T07:57:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.018+0000 7f1573e00640 1 -- start start 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.019+0000 7f1573e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1574143460 0x7f157415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.019+0000 7f1573e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f15740cde20 0x7f1574070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.019+0000 7f1573e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15740cc6b0 con 0x7f1574143930 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.019+0000 7f1573e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15740ce270 con 0x7f1574143460 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.019+0000 7f1573e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15740cc950 con 0x7f15740cde20 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1572a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1574143460 0x7f157415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1572a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1574143460 0x7f157415aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44016/0 (socket says 172.21.15.105:44016) 2024-03-23T07:57:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1572a00640 1 -- 172.21.15.105:0/2704950306 learned_addr learned my addr 172.21.15.105:0/2704950306 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1572a00640 1 -- 172.21.15.105:0/2704950306 >> v2:172.21.15.132:3300/0 conn(0x7f15740cde20 msgr2=0x7f1574070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1573400640 1 --2- 172.21.15.105:0/2704950306 >> v2:172.21.15.132:3300/0 conn(0x7f15740cde20 0x7f1574070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1572a00640 1 --2- 172.21.15.105:0/2704950306 >> v2:172.21.15.132:3300/0 conn(0x7f15740cde20 0x7f1574070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1572a00640 1 -- 172.21.15.105:0/2704950306 >> v1:172.21.15.132:6789/0 conn(0x7f1574143930 legacy=0x7f15740cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.020+0000 7f1572a00640 1 -- 172.21.15.105:0/2704950306 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740ccc30 con 0x7f1574143460 2024-03-23T07:57:58.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.021+0000 7f1572a00640 1 --2- 172.21.15.105:0/2704950306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143460 0x7f157415aba0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f15740b1a50 tx=0x7f156402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51c8dbbf6cbf9e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.021+0000 7f1571600640 1 -- 172.21.15.105:0/2704950306 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156402cb20 con 0x7f1574143460 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1571600640 1 -- 172.21.15.105:0/2704950306 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156402cc80 con 0x7f1574143460 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1571600640 1 -- 172.21.15.105:0/2704950306 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640343f0 con 0x7f1574143460 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 -- 172.21.15.105:0/2704950306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143460 msgr2=0x7f157415aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 --2- 172.21.15.105:0/2704950306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143460 0x7f157415aba0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f15740b1a50 tx=0x7f156402bf80 comp rx=0 tx=0).stop 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 -- 172.21.15.105:0/2704950306 shutdown_connections 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 --2- 172.21.15.105:0/2704950306 >> v2:172.21.15.132:3300/0 conn(0x7f15740cde20 0x7f1574070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 --2- 172.21.15.105:0/2704950306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143460 0x7f157415aba0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 -- 172.21.15.105:0/2704950306 >> 172.21.15.105:0/2704950306 conn(0x7f1574065df0 msgr2=0x7f1574069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 -- 172.21.15.105:0/2704950306 shutdown_connections 2024-03-23T07:57:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.022+0000 7f1573e00640 1 -- 172.21.15.105:0/2704950306 wait complete. 2024-03-23T07:57:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.024+0000 7f1573e00640 1 Processor -- start 2024-03-23T07:57:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.024+0000 7f1573e00640 1 -- start start 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15740cde20 0x7f15741f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1574143460 0x7f15740aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143930 0x7f157406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157415a370 con 0x7f1574143930 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15741f20a0 con 0x7f15740cde20 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15740ab9c0 con 0x7f1574143460 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143930 0x7f157406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143930 0x7f157406d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44020/0 (socket says 172.21.15.105:44020) 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1573400640 1 -- 172.21.15.105:0/327714365 learned_addr learned my addr 172.21.15.105:0/327714365 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.025+0000 7f1572a00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15740cde20 0x7f15741f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1572000640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1574143460 0x7f15740aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573400640 1 -- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1574143460 msgr2=0x7f15740aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573400640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1574143460 0x7f15740aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573400640 1 -- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15740cde20 msgr2=0x7f15741f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573400640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15740cde20 0x7f15741f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573400640 1 -- 172.21.15.105:0/327714365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564009b30 con 0x7f1574143930 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1572a00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15740cde20 0x7f15741f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1572000640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1574143460 0x7f15740aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573400640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143930 0x7f157406d7b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c7f0 tx=0x7f156c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f156aa00640 1 -- 172.21.15.105:0/327714365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c004220 con 0x7f1574143930 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f156aa00640 1 -- 172.21.15.105:0/327714365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c004380 con 0x7f1574143930 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f156aa00640 1 -- 172.21.15.105:0/327714365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c007860 con 0x7f1574143930 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15740abca0 con 0x7f1574143930 2024-03-23T07:57:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.026+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15740ac260 con 0x7f1574143930 2024-03-23T07:57:58.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.028+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157404e030 con 0x7f1574143930 2024-03-23T07:57:58.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.028+0000 7f156aa00640 1 -- 172.21.15.105:0/327714365 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f156c007bd0 con 0x7f1574143930 2024-03-23T07:57:58.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.029+0000 7f156aa00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac00 0x7f155406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.029+0000 7f156aa00640 1 -- 172.21.15.105:0/327714365 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f156c0893b0 con 0x7f1574143930 2024-03-23T07:57:58.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.030+0000 7f1572a00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac00 0x7f155406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.030+0000 7f1572a00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac00 0x7f155406d0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1564035000 tx=0x7f1564030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:58.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.034+0000 7f156aa00640 1 -- 172.21.15.105:0/327714365 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f156c0544c0 con 0x7f1574143930 2024-03-23T07:57:58.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.226+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15740cd1f0 con 0x7f155406ac00 2024-03-23T07:57:58.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.249+0000 7f156aa00640 1 -- 172.21.15.105:0/327714365 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286565 (secure 0 0 0) 0x7f15640343f0 con 0x7f155406ac00 2024-03-23T07:57:58.254 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:58.257 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:57:58.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.255+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac00 msgr2=0x7f155406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.255+0000 7f1573e00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac00 0x7f155406d0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1564035000 tx=0x7f1564030000 comp rx=0 tx=0).stop 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.255+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143930 msgr2=0x7f157406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.255+0000 7f1573e00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143930 0x7f157406d7b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c7f0 tx=0x7f156c00ccb0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 shutdown_connections 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574143930 0x7f157406d7b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f155406ac00 0x7f155406d0c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1574143460 0x7f15740aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 --2- 172.21.15.105:0/327714365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15740cde20 0x7f15741f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 >> 172.21.15.105:0/327714365 conn(0x7f1574065df0 msgr2=0x7f15740b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 shutdown_connections 2024-03-23T07:57:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.256+0000 7f1573e00640 1 -- 172.21.15.105:0/327714365 wait complete. 2024-03-23T07:57:58.275 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-03-23T07:57:58.075078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221036,"kb_used_data":4828,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753684,"statfs":{"total":773094113280,"available":772867772416,"internally_reserved":0,"allocated":4943872,"data_stored":2760056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001560"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.353468+0000","last_change":"2024-03-23T07:57:53.371775+0000","last_active":"2024-03-23T07:57:54.353468+0000","last_peered":"2024-03-23T07:57:54.353468+0000","last_clean":"2024-03-23T07:57:54.353468+0000","last_became_active":"2024-03-23T07:57:53.368696+0000","last_became_peered":"2024-03-23T07:57:53.368696+0000","last_unstale":"2024-03-23T07:57:54.353468+0000","last_undegraded":"2024-03-23T07:57:54.353468+0000","last_fullsized":"2024-03-23T07:57:54.353468+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:26:24.311051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212653+0000","last_change":"2024-03-23T07:57:53.369532+0000","last_active":"2024-03-23T07:57:55.212653+0000","last_peered":"2024-03-23T07:57:55.212653+0000","last_clean":"2024-03-23T07:57:55.212653+0000","last_became_active":"2024-03-23T07:57:53.366185+0000","last_became_peered":"2024-03-23T07:57:53.366185+0000","last_unstale":"2024-03-23T07:57:55.212653+0000","last_undegraded":"2024-03-23T07:57:55.212653+0000","last_fullsized":"2024-03-23T07:57:55.212653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:17:53.541393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.353371+0000","last_change":"2024-03-23T07:57:53.359138+0000","last_active":"2024-03-23T07:57:54.353371+0000","last_peered":"2024-03-23T07:57:54.353371+0000","last_clean":"2024-03-23T07:57:54.353371+0000","last_became_active":"2024-03-23T07:57:53.355869+0000","last_became_peered":"2024-03-23T07:57:53.355869+0000","last_unstale":"2024-03-23T07:57:54.353371+0000","last_undegraded":"2024-03-23T07:57:54.353371+0000","last_fullsized":"2024-03-23T07:57:54.353371+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:23.540254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352845+0000","last_change":"2024-03-23T07:57:53.356273+0000","last_active":"2024-03-23T07:57:54.352845+0000","last_peered":"2024-03-23T07:57:54.352845+0000","last_clean":"2024-03-23T07:57:54.352845+0000","last_became_active":"2024-03-23T07:57:53.355996+0000","last_became_peered":"2024-03-23T07:57:53.355996+0000","last_unstale":"2024-03-23T07:57:54.352845+0000","last_undegraded":"2024-03-23T07:57:54.352845+0000","last_fullsized":"2024-03-23T07:57:54.352845+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:50:22.766664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.801445+0000","last_change":"2024-03-23T07:57:53.377426+0000","last_active":"2024-03-23T07:57:54.801445+0000","last_peered":"2024-03-23T07:57:54.801445+0000","last_clean":"2024-03-23T07:57:54.801445+0000","last_became_active":"2024-03-23T07:57:53.375260+0000","last_became_peered":"2024-03-23T07:57:53.375260+0000","last_unstale":"2024-03-23T07:57:54.801445+0000","last_undegraded":"2024-03-23T07:57:54.801445+0000","last_fullsized":"2024-03-23T07:57:54.801445+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:34:32.442847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212181+0000","last_change":"2024-03-23T07:57:53.373534+0000","last_active":"2024-03-23T07:57:55.212181+0000","last_peered":"2024-03-23T07:57:55.212181+0000","last_clean":"2024-03-23T07:57:55.212181+0000","last_became_active":"2024-03-23T07:57:53.370792+0000","last_became_peered":"2024-03-23T07:57:53.370792+0000","last_unstale":"2024-03-23T07:57:55.212181+0000","last_undegraded":"2024-03-23T07:57:55.212181+0000","last_fullsized":"2024-03-23T07:57:55.212181+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:44:42.814154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508341+0000","last_change":"2024-03-23T07:57:53.360889+0000","last_active":"2024-03-23T07:57:54.508341+0000","last_peered":"2024-03-23T07:57:54.508341+0000","last_clean":"2024-03-23T07:57:54.508341+0000","last_became_active":"2024-03-23T07:57:53.360597+0000","last_became_peered":"2024-03-23T07:57:53.360597+0000","last_unstale":"2024-03-23T07:57:54.508341+0000","last_undegraded":"2024-03-23T07:57:54.508341+0000","last_fullsized":"2024-03-23T07:57:54.508341+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:34:32.442847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351051+0000","last_change":"2024-03-23T07:57:53.365178+0000","last_active":"2024-03-23T07:57:54.351051+0000","last_peered":"2024-03-23T07:57:54.351051+0000","last_clean":"2024-03-23T07:57:54.351051+0000","last_became_active":"2024-03-23T07:57:53.364812+0000","last_became_peered":"2024-03-23T07:57:53.364812+0000","last_unstale":"2024-03-23T07:57:54.351051+0000","last_undegraded":"2024-03-23T07:57:54.351051+0000","last_fullsized":"2024-03-23T07:57:54.351051+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:13.962310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.802220+0000","last_change":"2024-03-23T07:57:53.375238+0000","last_active":"2024-03-23T07:57:54.802220+0000","last_peered":"2024-03-23T07:57:54.802220+0000","last_clean":"2024-03-23T07:57:54.802220+0000","last_became_active":"2024-03-23T07:57:53.373980+0000","last_became_peered":"2024-03-23T07:57:53.373980+0000","last_unstale":"2024-03-23T07:57:54.802220+0000","last_undegraded":"2024-03-23T07:57:54.802220+0000","last_fullsized":"2024-03-23T07:57:54.802220+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:20.138687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508813+0000","last_change":"2024-03-23T07:57:53.350970+0000","last_active":"2024-03-23T07:57:54.508813+0000","last_peered":"2024-03-23T07:57:54.508813+0000","last_clean":"2024-03-23T07:57:54.508813+0000","last_became_active":"2024-03-23T07:57:53.350405+0000","last_became_peered":"2024-03-23T07:57:53.350405+0000","last_unstale":"2024-03-23T07:57:54.508813+0000","last_undegraded":"2024-03-23T07:57:54.508813+0000","last_fullsized":"2024-03-23T07:57:54.508813+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:20.138687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.807682+0000","last_change":"2024-03-23T07:57:53.362233+0000","last_active":"2024-03-23T07:57:54.807682+0000","last_peered":"2024-03-23T07:57:54.807682+0000","last_clean":"2024-03-23T07:57:54.807682+0000","last_became_active":"2024-03-23T07:57:53.360168+0000","last_became_peered":"2024-03-23T07:57:53.360168+0000","last_unstale":"2024-03-23T07:57:54.807682+0000","last_undegraded":"2024-03-23T07:57:54.807682+0000","last_fullsized":"2024-03-23T07:57:54.807682+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T07:58:54.144300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.801760+0000","last_change":"2024-03-23T07:57:53.371558+0000","last_active":"2024-03-23T07:57:54.801760+0000","last_peered":"2024-03-23T07:57:54.801760+0000","last_clean":"2024-03-23T07:57:54.801760+0000","last_became_active":"2024-03-23T07:57:53.371281+0000","last_became_peered":"2024-03-23T07:57:53.371281+0000","last_unstale":"2024-03-23T07:57:54.801760+0000","last_undegraded":"2024-03-23T07:57:54.801760+0000","last_fullsized":"2024-03-23T07:57:54.801760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:13:53.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508165+0000","last_change":"2024-03-23T07:57:53.364807+0000","last_active":"2024-03-23T07:57:54.508165+0000","last_peered":"2024-03-23T07:57:54.508165+0000","last_clean":"2024-03-23T07:57:54.508165+0000","last_became_active":"2024-03-23T07:57:53.364485+0000","last_became_peered":"2024-03-23T07:57:53.364485+0000","last_unstale":"2024-03-23T07:57:54.508165+0000","last_undegraded":"2024-03-23T07:57:54.508165+0000","last_fullsized":"2024-03-23T07:57:54.508165+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:00:05.919801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212290+0000","last_change":"2024-03-23T07:57:53.342004+0000","last_active":"2024-03-23T07:57:55.212290+0000","last_peered":"2024-03-23T07:57:55.212290+0000","last_clean":"2024-03-23T07:57:55.212290+0000","last_became_active":"2024-03-23T07:57:53.341488+0000","last_became_peered":"2024-03-23T07:57:53.341488+0000","last_unstale":"2024-03-23T07:57:55.212290+0000","last_undegraded":"2024-03-23T07:57:55.212290+0000","last_fullsized":"2024-03-23T07:57:55.212290+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:16:11.538206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346105+0000","last_change":"2024-03-23T07:57:53.365121+0000","last_active":"2024-03-23T07:57:54.346105+0000","last_peered":"2024-03-23T07:57:54.346105+0000","last_clean":"2024-03-23T07:57:54.346105+0000","last_became_active":"2024-03-23T07:57:53.364847+0000","last_became_peered":"2024-03-23T07:57:53.364847+0000","last_unstale":"2024-03-23T07:57:54.346105+0000","last_undegraded":"2024-03-23T07:57:54.346105+0000","last_fullsized":"2024-03-23T07:57:54.346105+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:24.375975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508018+0000","last_change":"2024-03-23T07:57:53.369545+0000","last_active":"2024-03-23T07:57:54.508018+0000","last_peered":"2024-03-23T07:57:54.508018+0000","last_clean":"2024-03-23T07:57:54.508018+0000","last_became_active":"2024-03-23T07:57:53.367612+0000","last_became_peered":"2024-03-23T07:57:53.367612+0000","last_unstale":"2024-03-23T07:57:54.508018+0000","last_undegraded":"2024-03-23T07:57:54.508018+0000","last_fullsized":"2024-03-23T07:57:54.508018+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:27:44.898294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.347053+0000","last_change":"2024-03-23T07:57:53.350321+0000","last_active":"2024-03-23T07:57:54.347053+0000","last_peered":"2024-03-23T07:57:54.347053+0000","last_clean":"2024-03-23T07:57:54.347053+0000","last_became_active":"2024-03-23T07:57:53.349957+0000","last_became_peered":"2024-03-23T07:57:53.349957+0000","last_unstale":"2024-03-23T07:57:54.347053+0000","last_undegraded":"2024-03-23T07:57:54.347053+0000","last_fullsized":"2024-03-23T07:57:54.347053+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:02:37.168156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346777+0000","last_change":"2024-03-23T07:57:53.350202+0000","last_active":"2024-03-23T07:57:54.346777+0000","last_peered":"2024-03-23T07:57:54.346777+0000","last_clean":"2024-03-23T07:57:54.346777+0000","last_became_active":"2024-03-23T07:57:53.349825+0000","last_became_peered":"2024-03-23T07:57:53.349825+0000","last_unstale":"2024-03-23T07:57:54.346777+0000","last_undegraded":"2024-03-23T07:57:54.346777+0000","last_fullsized":"2024-03-23T07:57:54.346777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:19:43.398101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352583+0000","last_change":"2024-03-23T07:57:53.374646+0000","last_active":"2024-03-23T07:57:54.352583+0000","last_peered":"2024-03-23T07:57:54.352583+0000","last_clean":"2024-03-23T07:57:54.352583+0000","last_became_active":"2024-03-23T07:57:53.374380+0000","last_became_peered":"2024-03-23T07:57:53.374380+0000","last_unstale":"2024-03-23T07:57:54.352583+0000","last_undegraded":"2024-03-23T07:57:54.352583+0000","last_fullsized":"2024-03-23T07:57:54.352583+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:37:05.821253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.807151+0000","last_change":"2024-03-23T07:57:53.355543+0000","last_active":"2024-03-23T07:57:54.807151+0000","last_peered":"2024-03-23T07:57:54.807151+0000","last_clean":"2024-03-23T07:57:54.807151+0000","last_became_active":"2024-03-23T07:57:53.353441+0000","last_became_peered":"2024-03-23T07:57:53.353441+0000","last_unstale":"2024-03-23T07:57:54.807151+0000","last_undegraded":"2024-03-23T07:57:54.807151+0000","last_fullsized":"2024-03-23T07:57:54.807151+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:16:11.538206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.807178+0000","last_change":"2024-03-23T07:57:53.363894+0000","last_active":"2024-03-23T07:57:54.807178+0000","last_peered":"2024-03-23T07:57:54.807178+0000","last_clean":"2024-03-23T07:57:54.807178+0000","last_became_active":"2024-03-23T07:57:53.363644+0000","last_became_peered":"2024-03-23T07:57:53.363644+0000","last_unstale":"2024-03-23T07:57:54.807178+0000","last_undegraded":"2024-03-23T07:57:54.807178+0000","last_fullsized":"2024-03-23T07:57:54.807178+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:15.211099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.806947+0000","last_change":"2024-03-23T07:57:53.365765+0000","last_active":"2024-03-23T07:57:54.806947+0000","last_peered":"2024-03-23T07:57:54.806947+0000","last_clean":"2024-03-23T07:57:54.806947+0000","last_became_active":"2024-03-23T07:57:53.363136+0000","last_became_peered":"2024-03-23T07:57:53.363136+0000","last_unstale":"2024-03-23T07:57:54.806947+0000","last_undegraded":"2024-03-23T07:57:54.806947+0000","last_fullsized":"2024-03-23T07:57:54.806947+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:04:13.442654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346043+0000","last_change":"2024-03-23T07:57:53.364542+0000","last_active":"2024-03-23T07:57:54.346043+0000","last_peered":"2024-03-23T07:57:54.346043+0000","last_clean":"2024-03-23T07:57:54.346043+0000","last_became_active":"2024-03-23T07:57:53.358643+0000","last_became_peered":"2024-03-23T07:57:53.358643+0000","last_unstale":"2024-03-23T07:57:54.346043+0000","last_undegraded":"2024-03-23T07:57:54.346043+0000","last_fullsized":"2024-03-23T07:57:54.346043+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:01.366631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351043+0000","last_change":"2024-03-23T07:57:53.352109+0000","last_active":"2024-03-23T07:57:54.351043+0000","last_peered":"2024-03-23T07:57:54.351043+0000","last_clean":"2024-03-23T07:57:54.351043+0000","last_became_active":"2024-03-23T07:57:53.351643+0000","last_became_peered":"2024-03-23T07:57:53.351643+0000","last_unstale":"2024-03-23T07:57:54.351043+0000","last_undegraded":"2024-03-23T07:57:54.351043+0000","last_fullsized":"2024-03-23T07:57:54.351043+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:27:06.176067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212956+0000","last_change":"2024-03-23T07:57:53.366332+0000","last_active":"2024-03-23T07:57:55.212956+0000","last_peered":"2024-03-23T07:57:55.212956+0000","last_clean":"2024-03-23T07:57:55.212956+0000","last_became_active":"2024-03-23T07:57:53.365454+0000","last_became_peered":"2024-03-23T07:57:53.365454+0000","last_unstale":"2024-03-23T07:57:55.212956+0000","last_undegraded":"2024-03-23T07:57:55.212956+0000","last_fullsized":"2024-03-23T07:57:55.212956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:15.211099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.353060+0000","last_change":"2024-03-23T07:57:53.360784+0000","last_active":"2024-03-23T07:57:54.353060+0000","last_peered":"2024-03-23T07:57:54.353060+0000","last_clean":"2024-03-23T07:57:54.353060+0000","last_became_active":"2024-03-23T07:57:53.356407+0000","last_became_peered":"2024-03-23T07:57:53.356407+0000","last_unstale":"2024-03-23T07:57:54.353060+0000","last_undegraded":"2024-03-23T07:57:54.353060+0000","last_fullsized":"2024-03-23T07:57:54.353060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:56:14.381992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.807731+0000","last_change":"2024-03-23T07:57:53.365338+0000","last_active":"2024-03-23T07:57:54.807731+0000","last_peered":"2024-03-23T07:57:54.807731+0000","last_clean":"2024-03-23T07:57:54.807731+0000","last_became_active":"2024-03-23T07:57:53.365114+0000","last_became_peered":"2024-03-23T07:57:53.365114+0000","last_unstale":"2024-03-23T07:57:54.807731+0000","last_undegraded":"2024-03-23T07:57:54.807731+0000","last_fullsized":"2024-03-23T07:57:54.807731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:28.415656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508423+0000","last_change":"2024-03-23T07:57:53.369547+0000","last_active":"2024-03-23T07:57:54.508423+0000","last_peered":"2024-03-23T07:57:54.508423+0000","last_clean":"2024-03-23T07:57:54.508423+0000","last_became_active":"2024-03-23T07:57:53.367516+0000","last_became_peered":"2024-03-23T07:57:53.367516+0000","last_unstale":"2024-03-23T07:57:54.508423+0000","last_undegraded":"2024-03-23T07:57:54.508423+0000","last_fullsized":"2024-03-23T07:57:54.508423+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:13:53.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346295+0000","last_change":"2024-03-23T07:57:53.367521+0000","last_active":"2024-03-23T07:57:54.346295+0000","last_peered":"2024-03-23T07:57:54.346295+0000","last_clean":"2024-03-23T07:57:54.346295+0000","last_became_active":"2024-03-23T07:57:53.367279+0000","last_became_peered":"2024-03-23T07:57:53.367279+0000","last_unstale":"2024-03-23T07:57:54.346295+0000","last_undegraded":"2024-03-23T07:57:54.346295+0000","last_fullsized":"2024-03-23T07:57:54.346295+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:18.424062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346227+0000","last_change":"2024-03-23T07:57:53.360520+0000","last_active":"2024-03-23T07:57:54.346227+0000","last_peered":"2024-03-23T07:57:54.346227+0000","last_clean":"2024-03-23T07:57:54.346227+0000","last_became_active":"2024-03-23T07:57:53.360277+0000","last_became_peered":"2024-03-23T07:57:53.360277+0000","last_unstale":"2024-03-23T07:57:54.346227+0000","last_undegraded":"2024-03-23T07:57:54.346227+0000","last_fullsized":"2024-03-23T07:57:54.346227+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:15:51.278023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508058+0000","last_change":"2024-03-23T07:57:53.362824+0000","last_active":"2024-03-23T07:57:54.508058+0000","last_peered":"2024-03-23T07:57:54.508058+0000","last_clean":"2024-03-23T07:57:54.508058+0000","last_became_active":"2024-03-23T07:57:53.362121+0000","last_became_peered":"2024-03-23T07:57:53.362121+0000","last_unstale":"2024-03-23T07:57:54.508058+0000","last_undegraded":"2024-03-23T07:57:54.508058+0000","last_fullsized":"2024-03-23T07:57:54.508058+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:02:05.769552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351356+0000","last_change":"2024-03-23T07:57:53.365192+0000","last_active":"2024-03-23T07:57:54.351356+0000","last_peered":"2024-03-23T07:57:54.351356+0000","last_clean":"2024-03-23T07:57:54.351356+0000","last_became_active":"2024-03-23T07:57:53.363833+0000","last_became_peered":"2024-03-23T07:57:53.363833+0000","last_unstale":"2024-03-23T07:57:54.351356+0000","last_undegraded":"2024-03-23T07:57:54.351356+0000","last_fullsized":"2024-03-23T07:57:54.351356+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:07.016805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346814+0000","last_change":"2024-03-23T07:57:53.343006+0000","last_active":"2024-03-23T07:57:54.346814+0000","last_peered":"2024-03-23T07:57:54.346814+0000","last_clean":"2024-03-23T07:57:54.346814+0000","last_became_active":"2024-03-23T07:57:53.342765+0000","last_became_peered":"2024-03-23T07:57:53.342765+0000","last_unstale":"2024-03-23T07:57:54.346814+0000","last_undegraded":"2024-03-23T07:57:54.346814+0000","last_fullsized":"2024-03-23T07:57:54.346814+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:05:13.962310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.369456+0000","last_change":"2024-03-23T07:57:52.892463+0000","last_active":"2024-03-23T07:57:53.369456+0000","last_peered":"2024-03-23T07:57:53.369456+0000","last_clean":"2024-03-23T07:57:53.369456+0000","last_became_active":"2024-03-23T07:57:52.892201+0000","last_became_peered":"2024-03-23T07:57:52.892201+0000","last_unstale":"2024-03-23T07:57:53.369456+0000","last_undegraded":"2024-03-23T07:57:53.369456+0000","last_fullsized":"2024-03-23T07:57:53.369456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:52.674138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212589+0000","last_change":"2024-03-23T07:57:53.365049+0000","last_active":"2024-03-23T07:57:55.212589+0000","last_peered":"2024-03-23T07:57:55.212589+0000","last_clean":"2024-03-23T07:57:55.212589+0000","last_became_active":"2024-03-23T07:57:53.364739+0000","last_became_peered":"2024-03-23T07:57:53.364739+0000","last_unstale":"2024-03-23T07:57:55.212589+0000","last_undegraded":"2024-03-23T07:57:55.212589+0000","last_fullsized":"2024-03-23T07:57:55.212589+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:24:15.858180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212782+0000","last_change":"2024-03-23T07:57:52.894911+0000","last_active":"2024-03-23T07:57:55.212782+0000","last_peered":"2024-03-23T07:57:55.212782+0000","last_clean":"2024-03-23T07:57:55.212782+0000","last_became_active":"2024-03-23T07:57:52.894603+0000","last_became_peered":"2024-03-23T07:57:52.894603+0000","last_unstale":"2024-03-23T07:57:55.212782+0000","last_undegraded":"2024-03-23T07:57:55.212782+0000","last_fullsized":"2024-03-23T07:57:55.212782+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:42:18.629286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.347091+0000","last_change":"2024-03-23T07:57:53.345459+0000","last_active":"2024-03-23T07:57:54.347091+0000","last_peered":"2024-03-23T07:57:54.347091+0000","last_clean":"2024-03-23T07:57:54.347091+0000","last_became_active":"2024-03-23T07:57:53.345209+0000","last_became_peered":"2024-03-23T07:57:53.345209+0000","last_unstale":"2024-03-23T07:57:54.347091+0000","last_undegraded":"2024-03-23T07:57:54.347091+0000","last_fullsized":"2024-03-23T07:57:54.347091+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:23:49.529129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.353572+0000","last_change":"2024-03-23T07:57:52.903651+0000","last_active":"2024-03-23T07:57:54.353572+0000","last_peered":"2024-03-23T07:57:54.353572+0000","last_clean":"2024-03-23T07:57:54.353572+0000","last_became_active":"2024-03-23T07:57:52.902651+0000","last_became_peered":"2024-03-23T07:57:52.902651+0000","last_unstale":"2024-03-23T07:57:54.353572+0000","last_undegraded":"2024-03-23T07:57:54.353572+0000","last_fullsized":"2024-03-23T07:57:54.353572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:27.400239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212901+0000","last_change":"2024-03-23T07:57:53.376036+0000","last_active":"2024-03-23T07:57:55.212901+0000","last_peered":"2024-03-23T07:57:55.212901+0000","last_clean":"2024-03-23T07:57:55.212901+0000","last_became_active":"2024-03-23T07:57:53.375746+0000","last_became_peered":"2024-03-23T07:57:53.375746+0000","last_unstale":"2024-03-23T07:57:55.212901+0000","last_undegraded":"2024-03-23T07:57:55.212901+0000","last_fullsized":"2024-03-23T07:57:55.212901+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:39:27.327672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.360305+0000","last_change":"2024-03-23T07:57:52.898253+0000","last_active":"2024-03-23T07:57:53.360305+0000","last_peered":"2024-03-23T07:57:53.360305+0000","last_clean":"2024-03-23T07:57:53.360305+0000","last_became_active":"2024-03-23T07:57:52.897946+0000","last_became_peered":"2024-03-23T07:57:52.897946+0000","last_unstale":"2024-03-23T07:57:53.360305+0000","last_undegraded":"2024-03-23T07:57:53.360305+0000","last_fullsized":"2024-03-23T07:57:53.360305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:04:12.427237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351605+0000","last_change":"2024-03-23T07:57:53.370756+0000","last_active":"2024-03-23T07:57:54.351605+0000","last_peered":"2024-03-23T07:57:54.351605+0000","last_clean":"2024-03-23T07:57:54.351605+0000","last_became_active":"2024-03-23T07:57:53.370380+0000","last_became_peered":"2024-03-23T07:57:53.370380+0000","last_unstale":"2024-03-23T07:57:54.351605+0000","last_undegraded":"2024-03-23T07:57:54.351605+0000","last_fullsized":"2024-03-23T07:57:54.351605+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:37:13.126192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.353281+0000","last_change":"2024-03-23T07:57:52.903306+0000","last_active":"2024-03-23T07:57:53.353281+0000","last_peered":"2024-03-23T07:57:53.353281+0000","last_clean":"2024-03-23T07:57:53.353281+0000","last_became_active":"2024-03-23T07:57:52.902521+0000","last_became_peered":"2024-03-23T07:57:52.902521+0000","last_unstale":"2024-03-23T07:57:53.353281+0000","last_undegraded":"2024-03-23T07:57:53.353281+0000","last_fullsized":"2024-03-23T07:57:53.353281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:26:33.909802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.806875+0000","last_change":"2024-03-23T07:57:53.365264+0000","last_active":"2024-03-23T07:57:54.806875+0000","last_peered":"2024-03-23T07:57:54.806875+0000","last_clean":"2024-03-23T07:57:54.806875+0000","last_became_active":"2024-03-23T07:57:53.365064+0000","last_became_peered":"2024-03-23T07:57:53.365064+0000","last_unstale":"2024-03-23T07:57:54.806875+0000","last_undegraded":"2024-03-23T07:57:54.806875+0000","last_fullsized":"2024-03-23T07:57:54.806875+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:31:35.908342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.370730+0000","last_change":"2024-03-23T07:57:52.894360+0000","last_active":"2024-03-23T07:57:53.370730+0000","last_peered":"2024-03-23T07:57:53.370730+0000","last_clean":"2024-03-23T07:57:53.370730+0000","last_became_active":"2024-03-23T07:57:52.894103+0000","last_became_peered":"2024-03-23T07:57:52.894103+0000","last_unstale":"2024-03-23T07:57:53.370730+0000","last_undegraded":"2024-03-23T07:57:53.370730+0000","last_fullsized":"2024-03-23T07:57:53.370730+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:15:33.069065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346117+0000","last_change":"2024-03-23T07:57:53.352694+0000","last_active":"2024-03-23T07:57:54.346117+0000","last_peered":"2024-03-23T07:57:54.346117+0000","last_clean":"2024-03-23T07:57:54.346117+0000","last_became_active":"2024-03-23T07:57:53.348763+0000","last_became_peered":"2024-03-23T07:57:53.348763+0000","last_unstale":"2024-03-23T07:57:54.346117+0000","last_undegraded":"2024-03-23T07:57:54.346117+0000","last_fullsized":"2024-03-23T07:57:54.346117+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:07:50.388631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351317+0000","last_change":"2024-03-23T07:57:52.900093+0000","last_active":"2024-03-23T07:57:54.351317+0000","last_peered":"2024-03-23T07:57:54.351317+0000","last_clean":"2024-03-23T07:57:54.351317+0000","last_became_active":"2024-03-23T07:57:52.899788+0000","last_became_peered":"2024-03-23T07:57:52.899788+0000","last_unstale":"2024-03-23T07:57:54.351317+0000","last_undegraded":"2024-03-23T07:57:54.351317+0000","last_fullsized":"2024-03-23T07:57:54.351317+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:42.255496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.213181+0000","last_change":"2024-03-23T07:57:53.368380+0000","last_active":"2024-03-23T07:57:55.213181+0000","last_peered":"2024-03-23T07:57:55.213181+0000","last_clean":"2024-03-23T07:57:55.213181+0000","last_became_active":"2024-03-23T07:57:53.368013+0000","last_became_peered":"2024-03-23T07:57:53.368013+0000","last_unstale":"2024-03-23T07:57:55.213181+0000","last_undegraded":"2024-03-23T07:57:55.213181+0000","last_fullsized":"2024-03-23T07:57:55.213181+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:00:58.522007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.347607+0000","last_change":"2024-03-23T07:57:52.893753+0000","last_active":"2024-03-23T07:57:53.347607+0000","last_peered":"2024-03-23T07:57:53.347607+0000","last_clean":"2024-03-23T07:57:53.347607+0000","last_became_active":"2024-03-23T07:57:52.893399+0000","last_became_peered":"2024-03-23T07:57:52.893399+0000","last_unstale":"2024-03-23T07:57:53.347607+0000","last_undegraded":"2024-03-23T07:57:53.347607+0000","last_fullsized":"2024-03-23T07:57:53.347607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:18:59.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352568+0000","last_change":"2024-03-23T07:57:53.355126+0000","last_active":"2024-03-23T07:57:54.352568+0000","last_peered":"2024-03-23T07:57:54.352568+0000","last_clean":"2024-03-23T07:57:54.352568+0000","last_became_active":"2024-03-23T07:57:53.353823+0000","last_became_peered":"2024-03-23T07:57:53.353823+0000","last_unstale":"2024-03-23T07:57:54.352568+0000","last_undegraded":"2024-03-23T07:57:54.352568+0000","last_fullsized":"2024-03-23T07:57:54.352568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:42:56.744368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.353413+0000","last_change":"2024-03-23T07:57:52.899885+0000","last_active":"2024-03-23T07:57:54.353413+0000","last_peered":"2024-03-23T07:57:54.353413+0000","last_clean":"2024-03-23T07:57:54.353413+0000","last_became_active":"2024-03-23T07:57:52.899572+0000","last_became_peered":"2024-03-23T07:57:52.899572+0000","last_unstale":"2024-03-23T07:57:54.353413+0000","last_undegraded":"2024-03-23T07:57:54.353413+0000","last_fullsized":"2024-03-23T07:57:54.353413+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:52.674138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212618+0000","last_change":"2024-03-23T07:57:53.349005+0000","last_active":"2024-03-23T07:57:55.212618+0000","last_peered":"2024-03-23T07:57:55.212618+0000","last_clean":"2024-03-23T07:57:55.212618+0000","last_became_active":"2024-03-23T07:57:53.348751+0000","last_became_peered":"2024-03-23T07:57:53.348751+0000","last_unstale":"2024-03-23T07:57:55.212618+0000","last_undegraded":"2024-03-23T07:57:55.212618+0000","last_fullsized":"2024-03-23T07:57:55.212618+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:44:42.814154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.332471+0000","last_change":"2024-03-23T07:57:52.898289+0000","last_active":"2024-03-23T07:57:53.332471+0000","last_peered":"2024-03-23T07:57:53.332471+0000","last_clean":"2024-03-23T07:57:53.332471+0000","last_became_active":"2024-03-23T07:57:52.896866+0000","last_became_peered":"2024-03-23T07:57:52.896866+0000","last_unstale":"2024-03-23T07:57:53.332471+0000","last_undegraded":"2024-03-23T07:57:53.332471+0000","last_fullsized":"2024-03-23T07:57:53.332471+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:49:40.406677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.353205+0000","last_change":"2024-03-23T07:57:53.365036+0000","last_active":"2024-03-23T07:57:54.353205+0000","last_peered":"2024-03-23T07:57:54.353205+0000","last_clean":"2024-03-23T07:57:54.353205+0000","last_became_active":"2024-03-23T07:57:53.364620+0000","last_became_peered":"2024-03-23T07:57:53.364620+0000","last_unstale":"2024-03-23T07:57:54.353205+0000","last_undegraded":"2024-03-23T07:57:54.353205+0000","last_fullsized":"2024-03-23T07:57:54.353205+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:49:41.422093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.375262+0000","last_change":"2024-03-23T07:57:52.903851+0000","last_active":"2024-03-23T07:57:53.375262+0000","last_peered":"2024-03-23T07:57:53.375262+0000","last_clean":"2024-03-23T07:57:53.375262+0000","last_became_active":"2024-03-23T07:57:52.903483+0000","last_became_peered":"2024-03-23T07:57:52.903483+0000","last_unstale":"2024-03-23T07:57:53.375262+0000","last_undegraded":"2024-03-23T07:57:53.375262+0000","last_fullsized":"2024-03-23T07:57:53.375262+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:10.577534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508787+0000","last_change":"2024-03-23T07:57:53.363127+0000","last_active":"2024-03-23T07:57:54.508787+0000","last_peered":"2024-03-23T07:57:54.508787+0000","last_clean":"2024-03-23T07:57:54.508787+0000","last_became_active":"2024-03-23T07:57:53.361897+0000","last_became_peered":"2024-03-23T07:57:53.361897+0000","last_unstale":"2024-03-23T07:57:54.508787+0000","last_undegraded":"2024-03-23T07:57:54.508787+0000","last_fullsized":"2024-03-23T07:57:54.508787+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:29:40.729537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352958+0000","last_change":"2024-03-23T07:57:52.896166+0000","last_active":"2024-03-23T07:57:54.352958+0000","last_peered":"2024-03-23T07:57:54.352958+0000","last_clean":"2024-03-23T07:57:54.352958+0000","last_became_active":"2024-03-23T07:57:52.893912+0000","last_became_peered":"2024-03-23T07:57:52.893912+0000","last_unstale":"2024-03-23T07:57:54.352958+0000","last_undegraded":"2024-03-23T07:57:54.352958+0000","last_fullsized":"2024-03-23T07:57:54.352958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:47:44.883295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351858+0000","last_change":"2024-03-23T07:57:53.356673+0000","last_active":"2024-03-23T07:57:54.351858+0000","last_peered":"2024-03-23T07:57:54.351858+0000","last_clean":"2024-03-23T07:57:54.351858+0000","last_became_active":"2024-03-23T07:57:53.356294+0000","last_became_peered":"2024-03-23T07:57:53.356294+0000","last_unstale":"2024-03-23T07:57:54.351858+0000","last_undegraded":"2024-03-23T07:57:54.351858+0000","last_fullsized":"2024-03-23T07:57:54.351858+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:19:33.471551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352231+0000","last_change":"2024-03-23T07:57:52.902210+0000","last_active":"2024-03-23T07:57:54.352231+0000","last_peered":"2024-03-23T07:57:54.352231+0000","last_clean":"2024-03-23T07:57:54.352231+0000","last_became_active":"2024-03-23T07:57:52.901929+0000","last_became_peered":"2024-03-23T07:57:52.901929+0000","last_unstale":"2024-03-23T07:57:54.352231+0000","last_undegraded":"2024-03-23T07:57:54.352231+0000","last_fullsized":"2024-03-23T07:57:54.352231+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:26:33.909802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212345+0000","last_change":"2024-03-23T07:57:53.361451+0000","last_active":"2024-03-23T07:57:55.212345+0000","last_peered":"2024-03-23T07:57:55.212345+0000","last_clean":"2024-03-23T07:57:55.212345+0000","last_became_active":"2024-03-23T07:57:53.361139+0000","last_became_peered":"2024-03-23T07:57:53.361139+0000","last_unstale":"2024-03-23T07:57:55.212345+0000","last_undegraded":"2024-03-23T07:57:55.212345+0000","last_fullsized":"2024-03-23T07:57:55.212345+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:20:02.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212344+0000","last_change":"2024-03-23T07:57:52.893037+0000","last_active":"2024-03-23T07:57:55.212344+0000","last_peered":"2024-03-23T07:57:55.212344+0000","last_clean":"2024-03-23T07:57:55.212344+0000","last_became_active":"2024-03-23T07:57:52.888134+0000","last_became_peered":"2024-03-23T07:57:52.888134+0000","last_unstale":"2024-03-23T07:57:55.212344+0000","last_undegraded":"2024-03-23T07:57:55.212344+0000","last_fullsized":"2024-03-23T07:57:55.212344+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:52.674138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.807474+0000","last_change":"2024-03-23T07:57:53.356401+0000","last_active":"2024-03-23T07:57:54.807474+0000","last_peered":"2024-03-23T07:57:54.807474+0000","last_clean":"2024-03-23T07:57:54.807474+0000","last_became_active":"2024-03-23T07:57:53.356168+0000","last_became_peered":"2024-03-23T07:57:53.356168+0000","last_unstale":"2024-03-23T07:57:54.807474+0000","last_undegraded":"2024-03-23T07:57:54.807474+0000","last_fullsized":"2024-03-23T07:57:54.807474+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:54.393224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.333088+0000","last_change":"2024-03-23T07:57:52.892341+0000","last_active":"2024-03-23T07:57:53.333088+0000","last_peered":"2024-03-23T07:57:53.333088+0000","last_clean":"2024-03-23T07:57:53.333088+0000","last_became_active":"2024-03-23T07:57:52.892049+0000","last_became_peered":"2024-03-23T07:57:52.892049+0000","last_unstale":"2024-03-23T07:57:53.333088+0000","last_undegraded":"2024-03-23T07:57:53.333088+0000","last_fullsized":"2024-03-23T07:57:53.333088+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:24:14.842764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.801406+0000","last_change":"2024-03-23T07:57:53.376726+0000","last_active":"2024-03-23T07:57:54.801406+0000","last_peered":"2024-03-23T07:57:54.801406+0000","last_clean":"2024-03-23T07:57:54.801406+0000","last_became_active":"2024-03-23T07:57:53.372160+0000","last_became_peered":"2024-03-23T07:57:53.372160+0000","last_unstale":"2024-03-23T07:57:54.801406+0000","last_undegraded":"2024-03-23T07:57:54.801406+0000","last_fullsized":"2024-03-23T07:57:54.801406+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:27:44.898294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.363030+0000","last_change":"2024-03-23T07:57:52.886981+0000","last_active":"2024-03-23T07:57:53.363030+0000","last_peered":"2024-03-23T07:57:53.363030+0000","last_clean":"2024-03-23T07:57:53.363030+0000","last_became_active":"2024-03-23T07:57:52.886714+0000","last_became_peered":"2024-03-23T07:57:52.886714+0000","last_unstale":"2024-03-23T07:57:53.363030+0000","last_undegraded":"2024-03-23T07:57:53.363030+0000","last_fullsized":"2024-03-23T07:57:53.363030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:42.255496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212559+0000","last_change":"2024-03-23T07:57:53.365272+0000","last_active":"2024-03-23T07:57:55.212559+0000","last_peered":"2024-03-23T07:57:55.212559+0000","last_clean":"2024-03-23T07:57:55.212559+0000","last_became_active":"2024-03-23T07:57:53.365007+0000","last_became_peered":"2024-03-23T07:57:53.365007+0000","last_unstale":"2024-03-23T07:57:55.212559+0000","last_undegraded":"2024-03-23T07:57:55.212559+0000","last_fullsized":"2024-03-23T07:57:55.212559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:34:01.821847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.358737+0000","last_change":"2024-03-23T07:57:52.900689+0000","last_active":"2024-03-23T07:57:53.358737+0000","last_peered":"2024-03-23T07:57:53.358737+0000","last_clean":"2024-03-23T07:57:53.358737+0000","last_became_active":"2024-03-23T07:57:52.899872+0000","last_became_peered":"2024-03-23T07:57:52.899872+0000","last_unstale":"2024-03-23T07:57:53.358737+0000","last_undegraded":"2024-03-23T07:57:53.358737+0000","last_fullsized":"2024-03-23T07:57:53.358737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:51:15.459650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508725+0000","last_change":"2024-03-23T07:57:53.370708+0000","last_active":"2024-03-23T07:57:54.508725+0000","last_peered":"2024-03-23T07:57:54.508725+0000","last_clean":"2024-03-23T07:57:54.508725+0000","last_became_active":"2024-03-23T07:57:53.370534+0000","last_became_peered":"2024-03-23T07:57:53.370534+0000","last_unstale":"2024-03-23T07:57:54.508725+0000","last_undegraded":"2024-03-23T07:57:54.508725+0000","last_fullsized":"2024-03-23T07:57:54.508725+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:34:08.583558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.371657+0000","last_change":"2024-03-23T07:57:52.904601+0000","last_active":"2024-03-23T07:57:53.371657+0000","last_peered":"2024-03-23T07:57:53.371657+0000","last_clean":"2024-03-23T07:57:53.371657+0000","last_became_active":"2024-03-23T07:57:52.904313+0000","last_became_peered":"2024-03-23T07:57:52.904313+0000","last_unstale":"2024-03-23T07:57:53.371657+0000","last_undegraded":"2024-03-23T07:57:53.371657+0000","last_fullsized":"2024-03-23T07:57:53.371657+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:36:42.132614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.508729+0000","last_change":"2024-03-23T07:57:53.362827+0000","last_active":"2024-03-23T07:57:54.508729+0000","last_peered":"2024-03-23T07:57:54.508729+0000","last_clean":"2024-03-23T07:57:54.508729+0000","last_became_active":"2024-03-23T07:57:53.362015+0000","last_became_peered":"2024-03-23T07:57:53.362015+0000","last_unstale":"2024-03-23T07:57:54.508729+0000","last_undegraded":"2024-03-23T07:57:54.508729+0000","last_fullsized":"2024-03-23T07:57:54.508729+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:41:44.568383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.801704+0000","last_change":"2024-03-23T07:57:52.907349+0000","last_active":"2024-03-23T07:57:54.801704+0000","last_peered":"2024-03-23T07:57:54.801704+0000","last_clean":"2024-03-23T07:57:54.801704+0000","last_became_active":"2024-03-23T07:57:52.907050+0000","last_became_peered":"2024-03-23T07:57:52.907050+0000","last_unstale":"2024-03-23T07:57:54.801704+0000","last_undegraded":"2024-03-23T07:57:54.801704+0000","last_fullsized":"2024-03-23T07:57:54.801704+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:42:27.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352855+0000","last_change":"2024-03-23T07:57:53.376624+0000","last_active":"2024-03-23T07:57:54.352855+0000","last_peered":"2024-03-23T07:57:54.352855+0000","last_clean":"2024-03-23T07:57:54.352855+0000","last_became_active":"2024-03-23T07:57:53.376361+0000","last_became_peered":"2024-03-23T07:57:53.376361+0000","last_unstale":"2024-03-23T07:57:54.352855+0000","last_undegraded":"2024-03-23T07:57:54.352855+0000","last_fullsized":"2024-03-23T07:57:54.352855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:07:12.470373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352347+0000","last_change":"2024-03-23T07:57:52.888444+0000","last_active":"2024-03-23T07:57:54.352347+0000","last_peered":"2024-03-23T07:57:54.352347+0000","last_clean":"2024-03-23T07:57:54.352347+0000","last_became_active":"2024-03-23T07:57:52.888147+0000","last_became_peered":"2024-03-23T07:57:52.888147+0000","last_unstale":"2024-03-23T07:57:54.352347+0000","last_undegraded":"2024-03-23T07:57:54.352347+0000","last_fullsized":"2024-03-23T07:57:54.352347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:08:01.159974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.345882+0000","last_change":"2024-03-23T07:57:53.365543+0000","last_active":"2024-03-23T07:57:54.345882+0000","last_peered":"2024-03-23T07:57:54.345882+0000","last_clean":"2024-03-23T07:57:54.345882+0000","last_became_active":"2024-03-23T07:57:53.364791+0000","last_became_peered":"2024-03-23T07:57:53.364791+0000","last_unstale":"2024-03-23T07:57:54.345882+0000","last_undegraded":"2024-03-23T07:57:54.345882+0000","last_fullsized":"2024-03-23T07:57:54.345882+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:27:23.540254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352081+0000","last_change":"2024-03-23T07:57:52.901586+0000","last_active":"2024-03-23T07:57:54.352081+0000","last_peered":"2024-03-23T07:57:54.352081+0000","last_clean":"2024-03-23T07:57:54.352081+0000","last_became_active":"2024-03-23T07:57:52.901313+0000","last_became_peered":"2024-03-23T07:57:52.901313+0000","last_unstale":"2024-03-23T07:57:54.352081+0000","last_undegraded":"2024-03-23T07:57:54.352081+0000","last_fullsized":"2024-03-23T07:57:54.352081+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:49:25.759750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351082+0000","last_change":"2024-03-23T07:57:53.368360+0000","last_active":"2024-03-23T07:57:54.351082+0000","last_peered":"2024-03-23T07:57:54.351082+0000","last_clean":"2024-03-23T07:57:54.351082+0000","last_became_active":"2024-03-23T07:57:53.367685+0000","last_became_peered":"2024-03-23T07:57:53.367685+0000","last_unstale":"2024-03-23T07:57:54.351082+0000","last_undegraded":"2024-03-23T07:57:54.351082+0000","last_fullsized":"2024-03-23T07:57:54.351082+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:39:27.327672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212481+0000","last_change":"2024-03-23T07:57:52.895603+0000","last_active":"2024-03-23T07:57:55.212481+0000","last_peered":"2024-03-23T07:57:55.212481+0000","last_clean":"2024-03-23T07:57:55.212481+0000","last_became_active":"2024-03-23T07:57:52.895328+0000","last_became_peered":"2024-03-23T07:57:52.895328+0000","last_unstale":"2024-03-23T07:57:55.212481+0000","last_undegraded":"2024-03-23T07:57:55.212481+0000","last_fullsized":"2024-03-23T07:57:55.212481+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:53.377807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.806929+0000","last_change":"2024-03-23T07:57:53.360480+0000","last_active":"2024-03-23T07:57:54.806929+0000","last_peered":"2024-03-23T07:57:54.806929+0000","last_clean":"2024-03-23T07:57:54.806929+0000","last_became_active":"2024-03-23T07:57:53.360241+0000","last_became_peered":"2024-03-23T07:57:53.360241+0000","last_unstale":"2024-03-23T07:57:54.806929+0000","last_undegraded":"2024-03-23T07:57:54.806929+0000","last_fullsized":"2024-03-23T07:57:54.806929+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:43.270913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.347807+0000","last_change":"2024-03-23T07:57:52.893874+0000","last_active":"2024-03-23T07:57:53.347807+0000","last_peered":"2024-03-23T07:57:53.347807+0000","last_clean":"2024-03-23T07:57:53.347807+0000","last_became_active":"2024-03-23T07:57:52.893539+0000","last_became_peered":"2024-03-23T07:57:52.893539+0000","last_unstale":"2024-03-23T07:57:53.347807+0000","last_undegraded":"2024-03-23T07:57:53.347807+0000","last_fullsized":"2024-03-23T07:57:53.347807+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:28:10.882850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.351758+0000","last_change":"2024-03-23T07:57:53.364607+0000","last_active":"2024-03-23T07:57:54.351758+0000","last_peered":"2024-03-23T07:57:54.351758+0000","last_clean":"2024-03-23T07:57:54.351758+0000","last_became_active":"2024-03-23T07:57:53.364304+0000","last_became_peered":"2024-03-23T07:57:53.364304+0000","last_unstale":"2024-03-23T07:57:54.351758+0000","last_undegraded":"2024-03-23T07:57:54.351758+0000","last_fullsized":"2024-03-23T07:57:54.351758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:17:53.541393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212056+0000","last_change":"2024-03-23T07:57:52.902158+0000","last_active":"2024-03-23T07:57:55.212056+0000","last_peered":"2024-03-23T07:57:55.212056+0000","last_clean":"2024-03-23T07:57:55.212056+0000","last_became_active":"2024-03-23T07:57:52.901831+0000","last_became_peered":"2024-03-23T07:57:52.901831+0000","last_unstale":"2024-03-23T07:57:55.212056+0000","last_undegraded":"2024-03-23T07:57:55.212056+0000","last_fullsized":"2024-03-23T07:57:55.212056+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T14:35:42.255496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.801918+0000","last_change":"2024-03-23T07:57:53.371518+0000","last_active":"2024-03-23T07:57:54.801918+0000","last_peered":"2024-03-23T07:57:54.801918+0000","last_clean":"2024-03-23T07:57:54.801918+0000","last_became_active":"2024-03-23T07:57:53.367104+0000","last_became_peered":"2024-03-23T07:57:53.367104+0000","last_unstale":"2024-03-23T07:57:54.801918+0000","last_undegraded":"2024-03-23T07:57:54.801918+0000","last_fullsized":"2024-03-23T07:57:54.801918+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:34:08.583558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.372987+0000","last_change":"2024-03-23T07:57:52.901403+0000","last_active":"2024-03-23T07:57:53.372987+0000","last_peered":"2024-03-23T07:57:53.372987+0000","last_clean":"2024-03-23T07:57:53.372987+0000","last_became_active":"2024-03-23T07:57:52.901119+0000","last_became_peered":"2024-03-23T07:57:52.901119+0000","last_unstale":"2024-03-23T07:57:53.372987+0000","last_undegraded":"2024-03-23T07:57:53.372987+0000","last_fullsized":"2024-03-23T07:57:53.372987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:17:52.525976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.807865+0000","last_change":"2024-03-23T07:57:53.362355+0000","last_active":"2024-03-23T07:57:54.807865+0000","last_peered":"2024-03-23T07:57:54.807865+0000","last_clean":"2024-03-23T07:57:54.807865+0000","last_became_active":"2024-03-23T07:57:53.359823+0000","last_became_peered":"2024-03-23T07:57:53.359823+0000","last_unstale":"2024-03-23T07:57:54.807865+0000","last_undegraded":"2024-03-23T07:57:54.807865+0000","last_fullsized":"2024-03-23T07:57:54.807865+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T13:11:37.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.805534+0000","last_change":"2024-03-23T07:57:52.907350+0000","last_active":"2024-03-23T07:57:54.805534+0000","last_peered":"2024-03-23T07:57:54.805534+0000","last_clean":"2024-03-23T07:57:54.805534+0000","last_became_active":"2024-03-23T07:57:52.907053+0000","last_became_peered":"2024-03-23T07:57:52.907053+0000","last_unstale":"2024-03-23T07:57:54.805534+0000","last_undegraded":"2024-03-23T07:57:54.805534+0000","last_fullsized":"2024-03-23T07:57:54.805534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:43:12.900219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352607+0000","last_change":"2024-03-23T07:57:53.364875+0000","last_active":"2024-03-23T07:57:54.352607+0000","last_peered":"2024-03-23T07:57:54.352607+0000","last_clean":"2024-03-23T07:57:54.352607+0000","last_became_active":"2024-03-23T07:57:53.363916+0000","last_became_peered":"2024-03-23T07:57:53.363916+0000","last_unstale":"2024-03-23T07:57:54.352607+0000","last_undegraded":"2024-03-23T07:57:54.352607+0000","last_fullsized":"2024-03-23T07:57:54.352607+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:15:04.554973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.801724+0000","last_change":"2024-03-23T07:57:52.905957+0000","last_active":"2024-03-23T07:57:54.801724+0000","last_peered":"2024-03-23T07:57:54.801724+0000","last_clean":"2024-03-23T07:57:54.801724+0000","last_became_active":"2024-03-23T07:57:52.905631+0000","last_became_peered":"2024-03-23T07:57:52.905631+0000","last_unstale":"2024-03-23T07:57:54.801724+0000","last_undegraded":"2024-03-23T07:57:54.801724+0000","last_fullsized":"2024-03-23T07:57:54.801724+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:54:22.994869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.801345+0000","last_change":"2024-03-23T07:57:53.373645+0000","last_active":"2024-03-23T07:57:54.801345+0000","last_peered":"2024-03-23T07:57:54.801345+0000","last_clean":"2024-03-23T07:57:54.801345+0000","last_became_active":"2024-03-23T07:57:53.373034+0000","last_became_peered":"2024-03-23T07:57:53.373034+0000","last_unstale":"2024-03-23T07:57:54.801345+0000","last_undegraded":"2024-03-23T07:57:54.801345+0000","last_fullsized":"2024-03-23T07:57:54.801345+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:29:40.729537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352274+0000","last_change":"2024-03-23T07:57:52.901718+0000","last_active":"2024-03-23T07:57:54.352274+0000","last_peered":"2024-03-23T07:57:54.352274+0000","last_clean":"2024-03-23T07:57:54.352274+0000","last_became_active":"2024-03-23T07:57:52.900719+0000","last_became_peered":"2024-03-23T07:57:52.900719+0000","last_unstale":"2024-03-23T07:57:54.352274+0000","last_undegraded":"2024-03-23T07:57:54.352274+0000","last_fullsized":"2024-03-23T07:57:54.352274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:04:12.427237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:55.212679+0000","last_change":"2024-03-23T07:57:53.361440+0000","last_active":"2024-03-23T07:57:55.212679+0000","last_peered":"2024-03-23T07:57:55.212679+0000","last_clean":"2024-03-23T07:57:55.212679+0000","last_became_active":"2024-03-23T07:57:53.361088+0000","last_became_peered":"2024-03-23T07:57:53.361088+0000","last_unstale":"2024-03-23T07:57:55.212679+0000","last_undegraded":"2024-03-23T07:57:55.212679+0000","last_fullsized":"2024-03-23T07:57:55.212679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T10:03:11.592951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.340939+0000","last_change":"2024-03-23T07:57:52.894480+0000","last_active":"2024-03-23T07:57:53.340939+0000","last_peered":"2024-03-23T07:57:53.340939+0000","last_clean":"2024-03-23T07:57:53.340939+0000","last_became_active":"2024-03-23T07:57:52.894218+0000","last_became_peered":"2024-03-23T07:57:52.894218+0000","last_unstale":"2024-03-23T07:57:53.340939+0000","last_undegraded":"2024-03-23T07:57:53.340939+0000","last_fullsized":"2024-03-23T07:57:53.340939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:10:19.123270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346225+0000","last_change":"2024-03-23T07:57:53.356113+0000","last_active":"2024-03-23T07:57:54.346225+0000","last_peered":"2024-03-23T07:57:54.346225+0000","last_clean":"2024-03-23T07:57:54.346225+0000","last_became_active":"2024-03-23T07:57:53.355888+0000","last_became_peered":"2024-03-23T07:57:53.355888+0000","last_unstale":"2024-03-23T07:57:54.346225+0000","last_undegraded":"2024-03-23T07:57:54.346225+0000","last_fullsized":"2024-03-23T07:57:54.346225+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:54:12.750892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.333186+0000","last_change":"2024-03-23T07:57:52.900055+0000","last_active":"2024-03-23T07:57:53.333186+0000","last_peered":"2024-03-23T07:57:53.333186+0000","last_clean":"2024-03-23T07:57:53.333186+0000","last_became_active":"2024-03-23T07:57:52.899309+0000","last_became_peered":"2024-03-23T07:57:52.899309+0000","last_unstale":"2024-03-23T07:57:53.333186+0000","last_undegraded":"2024-03-23T07:57:53.333186+0000","last_fullsized":"2024-03-23T07:57:53.333186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:39:26.312255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.807288+0000","last_change":"2024-03-23T07:57:53.373654+0000","last_active":"2024-03-23T07:57:54.807288+0000","last_peered":"2024-03-23T07:57:54.807288+0000","last_clean":"2024-03-23T07:57:54.807288+0000","last_became_active":"2024-03-23T07:57:53.373383+0000","last_became_peered":"2024-03-23T07:57:53.373383+0000","last_unstale":"2024-03-23T07:57:54.807288+0000","last_undegraded":"2024-03-23T07:57:54.807288+0000","last_fullsized":"2024-03-23T07:57:54.807288+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T09:08:08.207005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.352268+0000","last_change":"2024-03-23T07:57:52.899256+0000","last_active":"2024-03-23T07:57:54.352268+0000","last_peered":"2024-03-23T07:57:54.352268+0000","last_clean":"2024-03-23T07:57:54.352268+0000","last_became_active":"2024-03-23T07:57:52.898980+0000","last_became_peered":"2024-03-23T07:57:52.898980+0000","last_unstale":"2024-03-23T07:57:54.352268+0000","last_undegraded":"2024-03-23T07:57:54.352268+0000","last_fullsized":"2024-03-23T07:57:54.352268+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T11:33:53.377807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T07:57:54.346859+0000","last_change":"2024-03-23T07:57:53.359518+0000","last_active":"2024-03-23T07:57:54.346859+0000","last_peered":"2024-03-23T07:57:54.346859+0000","last_clean":"2024-03-23T07:57:54.346859+0000","last_became_active":"2024-03-23T07:57:53.359227+0000","last_became_peered":"2024-03-23T07:57:53.359227+0000","last_unstale":"2024-03-23T07:57:54.346859+0000","last_undegraded":"2024-03-23T07:57:54.346859+0000","last_fullsized":"2024-03-23T07:57:54.346859+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:52.848837+0000","last_clean_scrub_stamp":"2024-03-23T07:57:52.848837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:43:34.425091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.345183+0000","last_change":"2024-03-23T07:57:52.902897+0000","last_active":"2024-03-23T07:57:53.345183+0000","last_peered":"2024-03-23T07:57:53.345183+0000","last_clean":"2024-03-23T07:57:53.345183+0000","last_became_active":"2024-03-23T07:57:52.902578+0000","last_became_peered":"2024-03-23T07:57:52.902578+0000","last_unstale":"2024-03-23T07:57:53.345183+0000","last_undegraded":"2024-03-23T07:57:53.345183+0000","last_fullsized":"2024-03-23T07:57:53.345183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:57:51.833420+0000","last_clean_scrub_stamp":"2024-03-23T07:57:51.833420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:29:39.714121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-23T07:57:53.347698+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:57:53.347698+0000","last_peered":"2024-03-23T07:57:53.347698+0000","last_clean":"2024-03-23T07:57:53.347698+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:57:53.347698+0000","last_undegraded":"2024-03-23T07:57:53.347698+0000","last_fullsized":"2024-03-23T07:57:53.347698+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.006},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.67100000000000004}]},{"osd":1,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.236}]},{"osd":2,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.129}]},{"osd":4,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.0820000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":1.304},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.0449999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.202},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":3.6899999999999999,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.379}]},{"osd":7,"last update":"Sat Mar 23 07:57:11 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.3520000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476761,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":199798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.503},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.36599999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":0.55300000000000005},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.56699999999999995}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.59199999999999997}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.3420000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":0.871}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8410000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.573,"15min":1.573},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.75700000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":788042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.27,"5min":3.27,"15min":3.27},"last":0.84999999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.1699999999999999,"5min":3.1699999999999999,"15min":3.1699999999999999},"last":0.69299999999999995}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.1240000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":5.8109999999999999,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":1.419},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.3180000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.6060000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.212},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.579}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":1.516}]}]},{"osd":0,"up_from":16,"seq":68719476763,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":197744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.42399999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.40100000000000002}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.73799999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.872}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.0589999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.228},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.6459999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1299999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.3819999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":1.006,"5min":1.006,"15min":1.006},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.3360000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.427},{"interface":"front","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.772}]}]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.3480000000000001}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.306}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.875},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.0740000000000001}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.2350000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4810000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.6819999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.635}]}]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":199340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.579},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.21}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.157},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.651}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.2569999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.389},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.538}]},{"osd":5,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.337},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.999}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.6080000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.4850000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.60499999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.80000000000000004}]},{"osd":4,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Sat Mar 23 07:57:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.605,"5min":1.605,"15min":1.605},"min":{"1min":1.29,"5min":1.29,"15min":1.29},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.94399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:57:58.280 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T07:57:58.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T07:57:58.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.535+0000 7f2a97200640 1 Processor -- start 2024-03-23T07:57:58.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.536+0000 7f2a97200640 1 -- start start 2024-03-23T07:57:58.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.537+0000 7f2a97200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2a98143460 0x7f2a9815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.537+0000 7f2a97200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2a980cde20 0x7f2a98070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.537+0000 7f2a97200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a980cc6b0 con 0x7f2a98143930 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.537+0000 7f2a97200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980ce270 con 0x7f2a98143460 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.537+0000 7f2a97200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980cc950 con 0x7f2a980cde20 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.538+0000 7f2a95e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2a98143460 0x7f2a9815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.538+0000 7f2a95e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2a98143460 0x7f2a9815aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44036/0 (socket says 172.21.15.105:44036) 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.538+0000 7f2a95e00640 1 -- 172.21.15.105:0/3550790813 learned_addr learned my addr 172.21.15.105:0/3550790813 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.538+0000 7f2a95e00640 1 -- 172.21.15.105:0/3550790813 >> v2:172.21.15.132:3300/0 conn(0x7f2a980cde20 msgr2=0x7f2a98070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.538+0000 7f2a95e00640 1 --2- 172.21.15.105:0/3550790813 >> v2:172.21.15.132:3300/0 conn(0x7f2a980cde20 0x7f2a98070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.538+0000 7f2a95e00640 1 -- 172.21.15.105:0/3550790813 >> v1:172.21.15.105:6789/0 conn(0x7f2a98143930 legacy=0x7f2a980cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.538+0000 7f2a95e00640 1 -- 172.21.15.105:0/3550790813 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a980ccc30 con 0x7f2a98143460 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.539+0000 7f2a95e00640 1 --2- 172.21.15.105:0/3550790813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a9815aba0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a980b1a50 tx=0x7f2a8802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60a4e9fe539622b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.539+0000 7f2a94a00640 1 -- 172.21.15.105:0/3550790813 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8802f850 con 0x7f2a98143460 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a94a00640 1 -- 172.21.15.105:0/3550790813 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8802f9b0 con 0x7f2a98143460 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a94a00640 1 -- 172.21.15.105:0/3550790813 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8802fd40 con 0x7f2a98143460 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 -- 172.21.15.105:0/3550790813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 msgr2=0x7f2a9815aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 --2- 172.21.15.105:0/3550790813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a9815aba0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a980b1a50 tx=0x7f2a8802ba30 comp rx=0 tx=0).stop 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 -- 172.21.15.105:0/3550790813 shutdown_connections 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 --2- 172.21.15.105:0/3550790813 >> v2:172.21.15.132:3300/0 conn(0x7f2a980cde20 0x7f2a98070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 --2- 172.21.15.105:0/3550790813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a9815aba0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 -- 172.21.15.105:0/3550790813 >> 172.21.15.105:0/3550790813 conn(0x7f2a98065df0 msgr2=0x7f2a98069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 -- 172.21.15.105:0/3550790813 shutdown_connections 2024-03-23T07:57:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.540+0000 7f2a97200640 1 -- 172.21.15.105:0/3550790813 wait complete. 2024-03-23T07:57:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.542+0000 7f2a97200640 1 Processor -- start 2024-03-23T07:57:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.542+0000 7f2a97200640 1 -- start start 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a97200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a980cde20 0x7f2a98138150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a97200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a981f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a97200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a98143930 0x7f2a981f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a97200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9806db10 con 0x7f2a98143460 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a97200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9813b850 con 0x7f2a98143930 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a97200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9813bb30 con 0x7f2a980cde20 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a981f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a981f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44048/0 (socket says 172.21.15.105:44048) 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.543+0000 7f2a95400640 1 -- 172.21.15.105:0/732613414 learned_addr learned my addr 172.21.15.105:0/732613414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a95e00640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a980cde20 0x7f2a98138150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a95400640 1 -- 172.21.15.105:0/732613414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a980cde20 msgr2=0x7f2a98138150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a95400640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a980cde20 0x7f2a98138150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a95400640 1 -- 172.21.15.105:0/732613414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a98143930 msgr2=0x7f2a981f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a95400640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a98143930 0x7f2a981f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a95400640 1 -- 172.21.15.105:0/732613414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880094d0 con 0x7f2a98143460 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a95400640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a981f5840 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c400 tx=0x7f2a8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a8e000640 1 -- 172.21.15.105:0/732613414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a800052d0 con 0x7f2a98143460 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a8e000640 1 -- 172.21.15.105:0/732613414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a80005430 con 0x7f2a98143460 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a8e000640 1 -- 172.21.15.105:0/732613414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80003830 con 0x7f2a98143460 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9813be10 con 0x7f2a98143460 2024-03-23T07:57:58.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.544+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a98138690 con 0x7f2a98143460 2024-03-23T07:57:58.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.545+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9813c330 con 0x7f2a98143460 2024-03-23T07:57:58.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.546+0000 7f2a8e000640 1 -- 172.21.15.105:0/732613414 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2a80003990 con 0x7f2a98143460 2024-03-23T07:57:58.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.547+0000 7f2a8e000640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2a7406ac00 0x7f2a7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:58.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.547+0000 7f2a8e000640 1 -- 172.21.15.105:0/732613414 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2a80088ac0 con 0x7f2a98143460 2024-03-23T07:57:58.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.547+0000 7f2a95e00640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2a7406ac00 0x7f2a7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:58.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.548+0000 7f2a95e00640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2a7406ac00 0x7f2a7406d0c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a88036000 tx=0x7f2a88039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.552+0000 7f2a8e000640 1 -- 172.21.15.105:0/732613414 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2a9813c330 con 0x7f2a98143460 2024-03-23T07:57:58.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.816+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2a9813c330 con 0x7f2a98143460 2024-03-23T07:57:58.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.824+0000 7f2a8e000640 1 -- 172.21.15.105:0/732613414 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2a9813c330 con 0x7f2a98143460 2024-03-23T07:57:58.824 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2024-03-23T07:57:58.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2a7406ac00 msgr2=0x7f2a7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2a7406ac00 0x7f2a7406d0c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a88036000 tx=0x7f2a88039040 comp rx=0 tx=0).stop 2024-03-23T07:57:58.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 msgr2=0x7f2a981f5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:58.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a981f5840 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c400 tx=0x7f2a8000c8c0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 shutdown_connections 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a98143930 0x7f2a981f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a98143460 0x7f2a981f5840 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f2a7406ac00 0x7f2a7406d0c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 --2- 172.21.15.105:0/732613414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a980cde20 0x7f2a98138150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.827+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 >> 172.21.15.105:0/732613414 conn(0x7f2a98065df0 msgr2=0x7f2a9815d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.828+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 shutdown_connections 2024-03-23T07:57:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:58.828+0000 7f2a97200640 1 -- 172.21.15.105:0/732613414 wait complete. 2024-03-23T07:57:58.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:57:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.061+0000 7f5053e00640 1 Processor -- start 2024-03-23T07:57:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.061+0000 7f5053e00640 1 -- start start 2024-03-23T07:57:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5053e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5054160ca0 0x7f5054161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5053e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f505406c080 0x7f505406ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5053e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50540cda90 con 0x7f505406cf40 2024-03-23T07:57:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5053e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50540ce280 con 0x7f505406c080 2024-03-23T07:57:59.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5053e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505406d3b0 con 0x7f5054160ca0 2024-03-23T07:57:59.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5052a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5054160ca0 0x7f5054161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5052a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5054160ca0 0x7f5054161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:43708/0 (socket says 172.21.15.105:43708) 2024-03-23T07:57:59.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f504a000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f505406c080 0x7f505406ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.063+0000 7f5052a00640 1 -- 172.21.15.105:0/452349791 learned_addr learned my addr 172.21.15.105:0/452349791 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.064+0000 7f5052a00640 1 -- 172.21.15.105:0/452349791 >> v2:172.21.15.132:3300/0 conn(0x7f505406c080 msgr2=0x7f505406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.064+0000 7f5052a00640 1 --2- 172.21.15.105:0/452349791 >> v2:172.21.15.132:3300/0 conn(0x7f505406c080 0x7f505406ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.064+0000 7f5052a00640 1 -- 172.21.15.105:0/452349791 >> v1:172.21.15.132:6789/0 conn(0x7f505406cf40 legacy=0x7f5054070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.064+0000 7f5052a00640 1 -- 172.21.15.105:0/452349791 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054162220 con 0x7f5054160ca0 2024-03-23T07:57:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.066+0000 7f5052a00640 1 --2- 172.21.15.105:0/452349791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5054160ca0 0x7f5054161ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5044009870 tx=0x7f504402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fdb7bc925ca577d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.066+0000 7f5052000640 1 -- 172.21.15.105:0/452349791 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044031640 con 0x7f5054160ca0 2024-03-23T07:57:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.066+0000 7f5052000640 1 -- 172.21.15.105:0/452349791 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50440317a0 con 0x7f5054160ca0 2024-03-23T07:57:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.066+0000 7f5052000640 1 -- 172.21.15.105:0/452349791 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044031b30 con 0x7f5054160ca0 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 -- 172.21.15.105:0/452349791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5054160ca0 msgr2=0x7f5054161ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 --2- 172.21.15.105:0/452349791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5054160ca0 0x7f5054161ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5044009870 tx=0x7f504402b8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 -- 172.21.15.105:0/452349791 shutdown_connections 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 --2- 172.21.15.105:0/452349791 >> v2:172.21.15.132:3300/0 conn(0x7f505406c080 0x7f505406ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 --2- 172.21.15.105:0/452349791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5054160ca0 0x7f5054161ce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 -- 172.21.15.105:0/452349791 >> 172.21.15.105:0/452349791 conn(0x7f5054065df0 msgr2=0x7f5054069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 -- 172.21.15.105:0/452349791 shutdown_connections 2024-03-23T07:57:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.067+0000 7f5053e00640 1 -- 172.21.15.105:0/452349791 wait complete. 2024-03-23T07:57:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.069+0000 7f5053e00640 1 Processor -- start 2024-03-23T07:57:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.069+0000 7f5053e00640 1 -- start start 2024-03-23T07:57:59.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5053e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505406c080 0x7f505414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5053e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f505406cf40 0x7f50541fa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5053e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5054160ca0 0x7f505414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5053e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50541f9ef0 con 0x7f505406c080 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5053e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505414c600 con 0x7f5054160ca0 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5053e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50540ac5d0 con 0x7f505406cf40 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5052a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505406c080 0x7f505414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5052a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505406c080 0x7f505414c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44060/0 (socket says 172.21.15.105:44060) 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5052a00640 1 -- 172.21.15.105:0/133574134 learned_addr learned my addr 172.21.15.105:0/133574134 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f504a000640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f505406cf40 0x7f50541fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5052a00640 1 -- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f505406cf40 msgr2=0x7f50541fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.070+0000 7f5052a00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f505406cf40 0x7f50541fa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f5052a00640 1 -- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5054160ca0 msgr2=0x7f505414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f5053400640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5054160ca0 0x7f505414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f5052a00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5054160ca0 0x7f505414e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f5052a00640 1 -- 172.21.15.105:0/133574134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440094d0 con 0x7f505406c080 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f504a000640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f505406cf40 0x7f50541fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f5052a00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505406c080 0x7f505414c0c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f504402b930 tx=0x7f50440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f504b400640 1 -- 172.21.15.105:0/133574134 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504403a930 con 0x7f505406c080 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f504b400640 1 -- 172.21.15.105:0/133574134 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504403aa90 con 0x7f505406c080 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f504b400640 1 -- 172.21.15.105:0/133574134 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504403ae40 con 0x7f505406c080 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505414f860 con 0x7f505406c080 2024-03-23T07:57:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.071+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505414f580 con 0x7f505406c080 2024-03-23T07:57:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.072+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505404e030 con 0x7f505406c080 2024-03-23T07:57:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.073+0000 7f504b400640 1 -- 172.21.15.105:0/133574134 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f504402f050 con 0x7f505406c080 2024-03-23T07:57:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.074+0000 7f504b400640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f503006ac00 0x7f503006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.074+0000 7f504b400640 1 -- 172.21.15.105:0/133574134 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f50440ab2e0 con 0x7f505406c080 2024-03-23T07:57:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.074+0000 7f504a000640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f503006ac00 0x7f503006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.075+0000 7f504a000640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f503006ac00 0x7f503006d0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5054152f80 tx=0x7f503c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.078+0000 7f504b400640 1 -- 172.21.15.105:0/133574134 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5044076380 con 0x7f505406c080 2024-03-23T07:57:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.292+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50540a9760 con 0x7f505406c080 2024-03-23T07:57:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.293+0000 7f504b400640 1 -- 172.21.15.105:0/133574134 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f504402d040 con 0x7f505406c080 2024-03-23T07:57:59.293 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:57:59.293 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:57:56.345100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/3723879365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3723879365},{"type":"v1","addr":"172.21.15.105:6837","nonce":3723879365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:57:59.293 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 16 2024-03-23T07:57:59.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.295+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f503006ac00 msgr2=0x7f503006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.295+0000 7f5053e00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f503006ac00 0x7f503006d0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5054152f80 tx=0x7f503c008040 comp rx=0 tx=0).stop 2024-03-23T07:57:59.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.295+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505406c080 msgr2=0x7f505414c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.295+0000 7f5053e00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505406c080 0x7f505414c0c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f504402b930 tx=0x7f50440046c0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 shutdown_connections 2024-03-23T07:57:59.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5054160ca0 0x7f505414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f503006ac00 0x7f503006d0c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f505406cf40 0x7f50541fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 --2- 172.21.15.105:0/133574134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505406c080 0x7f505414c0c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 >> 172.21.15.105:0/133574134 conn(0x7f5054065df0 msgr2=0x7f50540aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 shutdown_connections 2024-03-23T07:57:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.296+0000 7f5053e00640 1 -- 172.21.15.105:0/133574134 wait complete. 2024-03-23T07:57:59.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T07:57:53.325549+0000', 'modified': '2024-03-23T07:57:56.345100+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5262}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5262': {'gid': 5262, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.105:6837/3723879365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3723879365}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3723879365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T07:57:59.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T07:57:59.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-23T07:57:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.940+0000 7f4331400640 1 Processor -- start 2024-03-23T07:57:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.940+0000 7f4331400640 1 -- start start 2024-03-23T07:57:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f4331400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f432c0cc330 0x7f432c0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f4331400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f432c0b1e10 0x7f432c0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f4331400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c15e010 con 0x7f432c159690 2024-03-23T07:57:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f4331400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c0caec0 con 0x7f432c0cc330 2024-03-23T07:57:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f4331400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c0cc7f0 con 0x7f432c0b1e10 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f432a800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f432c159690 0x7f432c0ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42374/0 (socket says 172.21.15.105:42374) 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f4321e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f432c0cc330 0x7f432c0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.941+0000 7f432a800640 1 -- 172.21.15.105:0/2028516009 learned_addr learned my addr 172.21.15.105:0/2028516009 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.942+0000 7f432b200640 1 --2- 172.21.15.105:0/2028516009 >> v2:172.21.15.132:3300/0 conn(0x7f432c0b1e10 0x7f432c0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.942+0000 7f4321e00640 1 -- 172.21.15.105:0/2028516009 >> v2:172.21.15.132:3300/0 conn(0x7f432c0b1e10 msgr2=0x7f432c0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.942+0000 7f4321e00640 1 --2- 172.21.15.105:0/2028516009 >> v2:172.21.15.132:3300/0 conn(0x7f432c0b1e10 0x7f432c0cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.942+0000 7f4321e00640 1 -- 172.21.15.105:0/2028516009 >> v1:172.21.15.105:6789/0 conn(0x7f432c159690 legacy=0x7f432c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.942+0000 7f4321e00640 1 -- 172.21.15.105:0/2028516009 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0ccad0 con 0x7f432c0cc330 2024-03-23T07:57:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.942+0000 7f4329e00640 1 -- 172.21.15.105:0/2028516009 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073401057 0 0) 0x7f432c15e010 con 0x7f432c159690 2024-03-23T07:57:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.943+0000 7f4321e00640 1 --2- 172.21.15.105:0/2028516009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0cc330 0x7f432c0b18d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f432c0addb0 tx=0x7f431402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3746bea72e8dc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.943+0000 7f4329e00640 1 -- 172.21.15.105:0/2028516009 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314031830 con 0x7f432c0cc330 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.943+0000 7f4329e00640 1 -- 172.21.15.105:0/2028516009 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314031990 con 0x7f432c0cc330 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.943+0000 7f4329e00640 1 -- 172.21.15.105:0/2028516009 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314031d20 con 0x7f432c0cc330 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 -- 172.21.15.105:0/2028516009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0cc330 msgr2=0x7f432c0b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 --2- 172.21.15.105:0/2028516009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0cc330 0x7f432c0b18d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f432c0addb0 tx=0x7f431402b8d0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 -- 172.21.15.105:0/2028516009 shutdown_connections 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 --2- 172.21.15.105:0/2028516009 >> v2:172.21.15.132:3300/0 conn(0x7f432c0b1e10 0x7f432c0cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 --2- 172.21.15.105:0/2028516009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0cc330 0x7f432c0b18d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 -- 172.21.15.105:0/2028516009 >> 172.21.15.105:0/2028516009 conn(0x7f432c06a0b0 msgr2=0x7f432c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 -- 172.21.15.105:0/2028516009 shutdown_connections 2024-03-23T07:57:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.944+0000 7f4331400640 1 -- 172.21.15.105:0/2028516009 wait complete. 2024-03-23T07:57:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.946+0000 7f4331400640 1 Processor -- start 2024-03-23T07:57:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.946+0000 7f4331400640 1 -- start start 2024-03-23T07:57:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f4331400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0b1e10 0x7f432c14eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f4331400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f432c0cc330 0x7f432c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f4331400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432c159690 0x7f432c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f4331400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c14f0f0 con 0x7f432c0b1e10 2024-03-23T07:57:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f4331400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c14f3d0 con 0x7f432c0cc330 2024-03-23T07:57:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f4331400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c14f6b0 con 0x7f432c159690 2024-03-23T07:57:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0b1e10 0x7f432c14eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0b1e10 0x7f432c14eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44078/0 (socket says 172.21.15.105:44078) 2024-03-23T07:57:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 -- 172.21.15.105:0/1513856882 learned_addr learned my addr 172.21.15.105:0/1513856882 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 -- 172.21.15.105:0/1513856882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432c159690 msgr2=0x7f432c15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432c159690 0x7f432c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 -- 172.21.15.105:0/1513856882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f432c0cc330 msgr2=0x7f432c0ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f432c0cc330 0x7f432c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.947+0000 7f432a800640 1 -- 172.21.15.105:0/1513856882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009400 con 0x7f432c0b1e10 2024-03-23T07:57:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.948+0000 7f432a800640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0b1e10 0x7f432c14eb10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c430 tx=0x7f431c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.948+0000 7f4323400640 1 -- 172.21.15.105:0/1513856882 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0074e0 con 0x7f432c0b1e10 2024-03-23T07:57:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.948+0000 7f4323400640 1 -- 172.21.15.105:0/1513856882 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c007660 con 0x7f432c0b1e10 2024-03-23T07:57:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.948+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c156d20 con 0x7f432c0b1e10 2024-03-23T07:57:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.948+0000 7f4323400640 1 -- 172.21.15.105:0/1513856882 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c004a20 con 0x7f432c0b1e10 2024-03-23T07:57:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.948+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c14f990 con 0x7f432c0b1e10 2024-03-23T07:57:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.950+0000 7f4323400640 1 -- 172.21.15.105:0/1513856882 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f431c004b80 con 0x7f432c0b1e10 2024-03-23T07:57:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.951+0000 7f4323400640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f430806ac30 0x7f430806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.951+0000 7f4323400640 1 -- 172.21.15.105:0/1513856882 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f431c0893a0 con 0x7f432c0b1e10 2024-03-23T07:57:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.951+0000 7f4321e00640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f430806ac30 0x7f430806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.952+0000 7f43341a1740 1 Processor -- start 2024-03-23T07:57:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.952+0000 7f4321e00640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f430806ac30 0x7f430806d0f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4314036000 tx=0x7f4314030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.952+0000 7f43341a1740 1 -- start start 2024-03-23T07:57:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be7163ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f43341a1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be7193a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71641e0 0x555be7193fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f43341a1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be713f2c0 con 0x555be71910e0 2024-03-23T07:57:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be70d48f0 con 0x555be7190d00 2024-03-23T07:57:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555be70d2080 con 0x555be71641e0 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f4321e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be7193a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f4321e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be7193a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44090/0 (socket says 172.21.15.105:44090) 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.953+0000 7f4321e00640 1 -- 172.21.15.105:0/3819606646 learned_addr learned my addr 172.21.15.105:0/3819606646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f4321e00640 1 -- 172.21.15.105:0/3819606646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71641e0 msgr2=0x555be7193fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f4321e00640 1 --2- 172.21.15.105:0/3819606646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71641e0 0x555be7193fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f4321e00640 1 -- 172.21.15.105:0/3819606646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 msgr2=0x555be7163ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f4321e00640 1 --2- 172.21.15.105:0/3819606646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be7163ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f4321e00640 1 -- 172.21.15.105:0/3819606646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555be7098a10 con 0x555be71910e0 2024-03-23T07:57:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f4321e00640 1 --2- 172.21.15.105:0/3819606646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be7193a90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f431403b530 tx=0x7f4314031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e0e530a84acc15 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f431b400640 1 -- 172.21.15.105:0/3819606646 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314004230 con 0x555be71910e0 2024-03-23T07:57:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f431b400640 1 -- 172.21.15.105:0/3819606646 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314004390 con 0x555be71910e0 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.954+0000 7f431b400640 1 -- 172.21.15.105:0/3819606646 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314003850 con 0x555be71910e0 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 -- 172.21.15.105:0/3819606646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 msgr2=0x555be7193a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 --2- 172.21.15.105:0/3819606646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be7193a90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f431403b530 tx=0x7f4314031000 comp rx=0 tx=0).stop 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 -- 172.21.15.105:0/3819606646 shutdown_connections 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 --2- 172.21.15.105:0/3819606646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71641e0 0x555be7193fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 --2- 172.21.15.105:0/3819606646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be7193a90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 --2- 172.21.15.105:0/3819606646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be7163ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 -- 172.21.15.105:0/3819606646 >> 172.21.15.105:0/3819606646 conn(0x555be7165f10 msgr2=0x555be71aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 -- 172.21.15.105:0/3819606646 shutdown_connections 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.955+0000 7f43341a1740 1 -- 172.21.15.105:0/3819606646 wait complete. 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.956+0000 7f43341a1740 1 -- start start 2024-03-23T07:57:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.956+0000 7f43341a1740 1 Processor -- start 2024-03-23T07:57:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be718f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71910e0 0x555be71964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f43341a1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be718fc40 0x555be71a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f43341a1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be70cd970 con 0x555be718fc40 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be706f0a0 con 0x555be7190d00 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555be706ed40 con 0x555be71910e0 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f43341a1740 10 client.0 _ll_register_callbacks cb 0x555be70c6fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f432b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be718fc40 0x555be71a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f432b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be718fc40 0x555be71a8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44102/0 (socket says 172.21.15.105:44102) 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f432b200640 1 -- 172.21.15.105:0/2308631671 learned_addr learned my addr 172.21.15.105:0/2308631671 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.957+0000 7f432b200640 1 -- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71910e0 msgr2=0x555be71964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f432b200640 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71910e0 0x555be71964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f432b200640 1 -- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 msgr2=0x555be718f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f432a800640 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be718f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f432b200640 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be718f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f432b200640 1 -- 172.21.15.105:0/2308631671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4314041070 con 0x555be718fc40 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f432a800640 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be718f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f432b200640 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be718fc40 0x555be71a8260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f432400c9e0 tx=0x7f432400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8850152462211c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f430fe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f430fe00640 1 -- 172.21.15.105:0/2308631671 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432400ac50 con 0x555be718fc40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f430fe00640 1 -- 172.21.15.105:0/2308631671 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4324012070 con 0x555be718fc40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555be70f7b10 con 0x555be718fc40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f430fe00640 1 -- 172.21.15.105:0/2308631671 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4324007810 con 0x555be718fc40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f43341a1740 10 client.5442 fetch_fsmap learned FSMap version 16 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.958+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x555be70cbff0 con 0x555be718fc40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f430fe00640 1 -- 172.21.15.105:0/2308631671 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555be70f7b10 con 0x555be718fc40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f430fe00640 1 -- 172.21.15.105:0/2308631671 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4324016420 con 0x555be718fc40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f43341a1740 10 client.5442 fetch_fsmap finished waiting for FSMap version 16 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f43341a1740 10 client.5442 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f43341a1740 20 client.5442 populate_metadata read hostname 'smithi105' 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f43341a1740 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be71a6a40 0x555be71cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f43341a1740 4 client.5442 mds_command: new command op to 5262 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 --> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555be6dc13c0 con 0x555be71a6a40 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.959+0000 7f432a800640 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be71a6a40 0x555be71cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.960+0000 7f430fe00640 10 client.5442 ms_handle_connect on v2:172.21.15.105:6836/3723879365 2024-03-23T07:57:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.960+0000 7f432a800640 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be71a6a40 0x555be71cbc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d22c25b5582b924 server_cookie=b09f54ab9664fe6f in_seq=0 out_seq=0 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.960+0000 7f430fe00640 1 -- 172.21.15.105:0/2308631671 <== mds.0 v2:172.21.15.105:6836/3723879365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555be6dc13c0 con 0x555be71a6a40 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.960+0000 7f430fe00640 10 client.5442 handle_command_reply: tid=0 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.960+0000 7f43341a1740 1 client.5442 shutdown 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be718fc40 msgr2=0x555be71a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be718fc40 0x555be71a8260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f432400c9e0 tx=0x7f432400cea0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 shutdown_connections 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be71a6a40 0x555be71cbc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be718fc40 0x555be71a8260 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f430fe00640 0 client.5442 ms_handle_reset on v2:172.21.15.105:6836/3723879365 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71910e0 0x555be71964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 --2- 172.21.15.105:0/2308631671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be7190d00 0x555be718f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 >> 172.21.15.105:0/2308631671 conn(0x555be71641e0 msgr2=0x555be7193d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 shutdown_connections 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 1 -- 172.21.15.105:0/2308631671 wait complete. 2024-03-23T07:57:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.961+0000 7f43341a1740 20 client.5442 trim_cache size 0 max 16384 2024-03-23T07:57:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.977+0000 7f43341a1740 1 Processor -- start 2024-03-23T07:57:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.977+0000 7f43341a1740 1 -- start start 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.977+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be7190d00 0x555be716af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.977+0000 7f43341a1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be71a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.977+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71af490 0x555be71a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.977+0000 7f43341a1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be6e34f20 con 0x555be71910e0 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be70c6930 con 0x555be71af490 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555be7196610 con 0x555be7190d00 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be71a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be71a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44112/0 (socket says 172.21.15.105:44112) 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 -- 172.21.15.105:0/2184288173 learned_addr learned my addr 172.21.15.105:0/2184288173 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 -- 172.21.15.105:0/2184288173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be7190d00 msgr2=0x555be716af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 --2- 172.21.15.105:0/2184288173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be7190d00 0x555be716af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 -- 172.21.15.105:0/2184288173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71af490 msgr2=0x555be71a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 --2- 172.21.15.105:0/2184288173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71af490 0x555be71a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 -- 172.21.15.105:0/2184288173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555be71017c0 con 0x555be71910e0 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.978+0000 7f4321e00640 1 --2- 172.21.15.105:0/2184288173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be71a8190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4314004000 tx=0x7f431402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd44a6f75754717 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f431be00640 1 -- 172.21.15.105:0/2184288173 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314055070 con 0x555be71910e0 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f431be00640 1 -- 172.21.15.105:0/2184288173 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314004030 con 0x555be71910e0 2024-03-23T07:57:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f431be00640 1 -- 172.21.15.105:0/2184288173 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314003830 con 0x555be71910e0 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 -- 172.21.15.105:0/2184288173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 msgr2=0x555be71a8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 --2- 172.21.15.105:0/2184288173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be71a8190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4314004000 tx=0x7f431402e000 comp rx=0 tx=0).stop 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 -- 172.21.15.105:0/2184288173 shutdown_connections 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 --2- 172.21.15.105:0/2184288173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71af490 0x555be71a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 --2- 172.21.15.105:0/2184288173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be71910e0 0x555be71a8190 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 --2- 172.21.15.105:0/2184288173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be7190d00 0x555be716af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 -- 172.21.15.105:0/2184288173 >> 172.21.15.105:0/2184288173 conn(0x555be71641e0 msgr2=0x555be71b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 -- 172.21.15.105:0/2184288173 shutdown_connections 2024-03-23T07:57:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.979+0000 7f43341a1740 1 -- 172.21.15.105:0/2184288173 wait complete. 2024-03-23T07:57:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.980+0000 7f43341a1740 1 -- start start 2024-03-23T07:57:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.980+0000 7f43341a1740 1 Processor -- start 2024-03-23T07:57:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f43341a1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be7190d00 0x555be720b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71910e0 0x555be71adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f43341a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71af490 0x555be71ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f43341a1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be71b2dd0 con 0x555be7190d00 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be720bab0 con 0x555be71910e0 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f43341a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555be71a5160 con 0x555be71af490 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f43341a1740 10 client.0 _ll_register_callbacks cb 0x555be71a8fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f432a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be7190d00 0x555be720b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f432a800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be7190d00 0x555be720b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44114/0 (socket says 172.21.15.105:44114) 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.981+0000 7f432a800640 1 -- 172.21.15.105:0/392008180 learned_addr learned my addr 172.21.15.105:0/392008180 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f432a800640 1 -- 172.21.15.105:0/392008180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71af490 msgr2=0x555be71ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f432a800640 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71af490 0x555be71ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f432a800640 1 -- 172.21.15.105:0/392008180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71910e0 msgr2=0x555be71adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f432a800640 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71910e0 0x555be71adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f432a800640 1 -- 172.21.15.105:0/392008180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555be7098a10 con 0x555be7190d00 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f432a800640 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be7190d00 0x555be720b570 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f431c057ce0 tx=0x7f431c00c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac558191923a2b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f430fe00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:57:59.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f430fe00640 1 -- 172.21.15.105:0/392008180 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0603d0 con 0x555be7190d00 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f430fe00640 1 -- 172.21.15.105:0/392008180 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f431c05e0b0 con 0x555be7190d00 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4314004390 con 0x555be7190d00 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f430fe00640 1 -- 172.21.15.105:0/392008180 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f431c0890c0 con 0x555be7190d00 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.982+0000 7f43341a1740 10 client.5448 fetch_fsmap learned FSMap version 16 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x555be706e2b0 con 0x555be7190d00 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f430fe00640 1 -- 172.21.15.105:0/392008180 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314004390 con 0x555be7190d00 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f430fe00640 1 -- 172.21.15.105:0/392008180 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f431c01b060 con 0x555be7190d00 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f43341a1740 10 client.5448 fetch_fsmap finished waiting for FSMap version 16 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f43341a1740 10 client.5448 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f43341a1740 20 client.5448 populate_metadata read hostname 'smithi105' 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f43341a1740 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be718d490 0x555be718a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f43341a1740 4 client.5448 mds_command: new command op to 5262 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 --> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555be6dd26c0 con 0x555be718d490 2024-03-23T07:57:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.983+0000 7f4321e00640 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be718d490 0x555be718a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f430fe00640 10 client.5448 ms_handle_connect on v2:172.21.15.105:6836/3723879365 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f4321e00640 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be718d490 0x555be718a620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7860942237f84c2 server_cookie=882334ce5e3504bb in_seq=0 out_seq=0 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f430fe00640 1 -- 172.21.15.105:0/392008180 <== mds.0 v2:172.21.15.105:6836/3723879365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x555be6dd26c0 con 0x555be718d490 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f430fe00640 10 client.5448 handle_command_reply: tid=0 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 client.5448 shutdown 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be7190d00 msgr2=0x555be720b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be7190d00 0x555be720b570 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f431c057ce0 tx=0x7f431c00c3b0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 shutdown_connections 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:6836/3723879365,v1:172.21.15.105:6837/3723879365] conn(0x555be718d490 0x555be718a620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555be71af490 0x555be71ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555be71910e0 0x555be71adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 --2- 172.21.15.105:0/392008180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555be7190d00 0x555be720b570 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 >> 172.21.15.105:0/392008180 conn(0x555be71641e0 msgr2=0x555be71a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f430fe00640 0 client.5448 ms_handle_reset on v2:172.21.15.105:6836/3723879365 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 shutdown_connections 2024-03-23T07:57:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 1 -- 172.21.15.105:0/392008180 wait complete. 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.984+0000 7f43341a1740 20 client.5448 trim_cache size 0 max 16384 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "e641b836-a0e5-4e95-93aa-bb0628868335", 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5262, 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 6.6546877999999996, 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 16, 2024-03-23T07:57:59.988 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 42, 2024-03-23T07:57:59.989 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 41, 2024-03-23T07:57:59.989 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 15.193049809 2024-03-23T07:57:59.989 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:57:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f430806ac30 msgr2=0x7f430806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f430806ac30 0x7f430806d0f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4314036000 tx=0x7f4314030000 comp rx=0 tx=0).stop 2024-03-23T07:57:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0b1e10 msgr2=0x7f432c14eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0b1e10 0x7f432c14eb10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c430 tx=0x7f431c00c8f0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 shutdown_connections 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f430806ac30 0x7f430806d0f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432c159690 0x7f432c15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f432c0cc330 0x7f432c0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 --2- 172.21.15.105:0/1513856882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f432c0b1e10 0x7f432c14eb10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 >> 172.21.15.105:0/1513856882 conn(0x7f432c06a0b0 msgr2=0x7f432c0b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 shutdown_connections 2024-03-23T07:57:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:57:59.985+0000 7f4331400640 1 -- 172.21.15.105:0/1513856882 wait complete. 2024-03-23T07:58:00.003 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T07:58:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.206+0000 7fb04b600640 1 Processor -- start 2024-03-23T07:58:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.207+0000 7fb04b600640 1 -- start start 2024-03-23T07:58:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb04b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415cd70 0x7fb0440d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb04b600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb0440d1330 0x7fb0440b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb04b600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0440b4c80 con 0x7fb0440b4880 2024-03-23T07:58:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb04b600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04415d150 con 0x7fb0440d1330 2024-03-23T07:58:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb04b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440d17d0 con 0x7fb04415cd70 2024-03-23T07:58:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb043e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb0440d1330 0x7fb0440b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb043e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb0440d1330 0x7fb0440b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44116/0 (socket says 172.21.15.105:44116) 2024-03-23T07:58:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb043e00640 1 -- 172.21.15.105:0/3083988769 learned_addr learned my addr 172.21.15.105:0/3083988769 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.208+0000 7fb048c00640 1 --2- 172.21.15.105:0/3083988769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415cd70 0x7fb0440d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.209+0000 7fb043e00640 1 -- 172.21.15.105:0/3083988769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415cd70 msgr2=0x7fb0440d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.209+0000 7fb043e00640 1 --2- 172.21.15.105:0/3083988769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415cd70 0x7fb0440d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.209+0000 7fb043e00640 1 -- 172.21.15.105:0/3083988769 >> v1:172.21.15.132:6789/0 conn(0x7fb0440b4880 legacy=0x7fb04406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.209+0000 7fb043e00640 1 -- 172.21.15.105:0/3083988769 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440d1ab0 con 0x7fb0440d1330 2024-03-23T07:58:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.210+0000 7fb043e00640 1 --2- 172.21.15.105:0/3083988769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0440b4340 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb034009e50 tx=0x7fb03402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd8da9f2e91f192 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.210+0000 7fb043400640 1 -- 172.21.15.105:0/3083988769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03402cb00 con 0x7fb0440d1330 2024-03-23T07:58:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.210+0000 7fb043400640 1 -- 172.21.15.105:0/3083988769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03402cc60 con 0x7fb0440d1330 2024-03-23T07:58:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.210+0000 7fb043400640 1 -- 172.21.15.105:0/3083988769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340343f0 con 0x7fb0440d1330 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 -- 172.21.15.105:0/3083988769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 msgr2=0x7fb0440b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 --2- 172.21.15.105:0/3083988769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0440b4340 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb034009e50 tx=0x7fb03402bf60 comp rx=0 tx=0).stop 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 -- 172.21.15.105:0/3083988769 shutdown_connections 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 --2- 172.21.15.105:0/3083988769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0440b4340 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 --2- 172.21.15.105:0/3083988769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415cd70 0x7fb0440d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 -- 172.21.15.105:0/3083988769 >> 172.21.15.105:0/3083988769 conn(0x7fb0440cf340 msgr2=0x7fb0440cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 -- 172.21.15.105:0/3083988769 shutdown_connections 2024-03-23T07:58:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.211+0000 7fb04b600640 1 -- 172.21.15.105:0/3083988769 wait complete. 2024-03-23T07:58:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.213+0000 7fb04b600640 1 Processor -- start 2024-03-23T07:58:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.213+0000 7fb04b600640 1 -- start start 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb04b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0440b4880 0x7fb0441f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb04b600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0441f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb04b600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04415cd70 0x7fb04414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb04b600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440c90b0 con 0x7fb0440d1330 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb04b600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440c9990 con 0x7fb04415cd70 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb04b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04414af10 con 0x7fb0440b4880 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb043e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0441f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb043e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0441f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44128/0 (socket says 172.21.15.105:44128) 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.214+0000 7fb043e00640 1 -- 172.21.15.105:0/2388064244 learned_addr learned my addr 172.21.15.105:0/2388064244 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb048c00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0440b4880 0x7fb0441f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb049600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04415cd70 0x7fb04414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb043e00640 1 -- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0440b4880 msgr2=0x7fb0441f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb043e00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0440b4880 0x7fb0441f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb043e00640 1 -- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04415cd70 msgr2=0x7fb04414c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb043e00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04415cd70 0x7fb04414c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb043e00640 1 -- 172.21.15.105:0/2388064244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034009b30 con 0x7fb0440d1330 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb049600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04415cd70 0x7fb04414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb048c00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0440b4880 0x7fb0441f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb043e00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0441f5910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb034000f80 tx=0x7fb034004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb040c00640 1 -- 172.21.15.105:0/2388064244 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034009510 con 0x7fb0440d1330 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb040c00640 1 -- 172.21.15.105:0/2388064244 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03403e450 con 0x7fb0440d1330 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb040c00640 1 -- 172.21.15.105:0/2388064244 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03403e800 con 0x7fb0440d1330 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440c8dd0 con 0x7fb0440d1330 2024-03-23T07:58:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.215+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04414aba0 con 0x7fb0440d1330 2024-03-23T07:58:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.217+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04414a720 con 0x7fb0440d1330 2024-03-23T07:58:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.218+0000 7fb040c00640 1 -- 172.21.15.105:0/2388064244 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb03403e960 con 0x7fb0440d1330 2024-03-23T07:58:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.218+0000 7fb040c00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb02006ac00 0x7fb02006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.218+0000 7fb040c00640 1 -- 172.21.15.105:0/2388064244 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fb0340ac310 con 0x7fb0440d1330 2024-03-23T07:58:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.219+0000 7fb048c00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb02006ac00 0x7fb02006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.219+0000 7fb048c00640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb02006ac00 0x7fb02006d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb02c006fd0 tx=0x7fb02c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.223+0000 7fb040c00640 1 -- 172.21.15.105:0/2388064244 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb04414a720 con 0x7fb0440d1330 2024-03-23T07:58:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.456+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb04414a720 con 0x7fb0440d1330 2024-03-23T07:58:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.456+0000 7fb040c00640 1 -- 172.21.15.105:0/2388064244 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb04414a720 con 0x7fb0440d1330 2024-03-23T07:58:00.456 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb02006ac00 msgr2=0x7fb02006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb02006ac00 0x7fb02006d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb02c006fd0 tx=0x7fb02c008040 comp rx=0 tx=0).stop 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 msgr2=0x7fb0441f5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0441f5910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb034000f80 tx=0x7fb034004a40 comp rx=0 tx=0).stop 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 shutdown_connections 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04415cd70 0x7fb04414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0440d1330 0x7fb0441f5910 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb02006ac00 0x7fb02006d0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 --2- 172.21.15.105:0/2388064244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0440b4880 0x7fb0441f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 >> 172.21.15.105:0/2388064244 conn(0x7fb0440cf340 msgr2=0x7fb0440b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 shutdown_connections 2024-03-23T07:58:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:00.459+0000 7fb04b600640 1 -- 172.21.15.105:0/2388064244 wait complete. 2024-03-23T07:58:00.476 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-23T07:58:00.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T07:58:00.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-23T07:58:00.476 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T07:58:00.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T07:58:00.477 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-23T07:58:00.477 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-23T07:58:00.477 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T07:58:00.477 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T07:58:00.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:00.477 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:58:00.527 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42445sec preferred_lft 42445sec 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 62:fb:8d:27:26:ba brd ff:ff:ff:ff:ff:ff 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::88e7:17ff:fe0f:aec1/64 scope link 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T07:58:00.528 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether de:d6:9d:2d:cf:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T07:58:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::dcd6:9dff:fe2d:cfcd/64 scope link 2024-03-23T07:58:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:58:00.529 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T07:58:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 62:fb:8d:27:26:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T07:58:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::60fb:8dff:fe27:26ba/64 scope link 2024-03-23T07:58:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-23T07:58:00.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:00.530 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T07:58:00.616 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-23T07:58:00.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:00.621 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-23T07:58:00.672 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-23T07:58:00.672 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T07:58:00.672 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T07:58:00.672 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T07:58:00.672 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T07:58:00.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:00.672 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-23T07:58:00.672 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-23T07:58:00.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T07:58:00.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T07:58:00.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T07:58:00.673 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-23T07:58:00.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:00.855 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:58:00.899 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T07:58:00.899 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T07:58:00.900 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T07:58:00.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:00.901 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-03-23T07:58:01.036 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:58:01.036 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T07:58:01.085 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T07:58:01.086 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T07:58:01.086 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T07:58:01.134 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-23T07:58:01.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:01.135 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:58:01.271 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:58:01.271 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:58:01.305 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-23T07:58:01.306 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-23T07:58:01.306 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-03-23T07:58:01.306 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 0@e641b836-a0e5-4e95-93aa-bb0628868335.cephfs=/ 2024-03-23T07:58:01.306 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.105:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2024-03-23T07:58:01.307 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-23T07:58:01.308 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-03-23T07:58:01.308 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T07:58:01.308 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T07:58:01.308 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T07:58:01.311 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T07:58:01.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:01.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T07:58:01.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:01.396 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T07:58:01.461 INFO:teuthology.orchestra.run.smithi105.stdout:client4891 2024-03-23T07:58:01.462 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:58:01.462 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:01.513 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T07:58:01.513 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-03-23T07:58:01.513 DEBUG:teuthology.orchestra.run.smithi105:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-03-23T07:58:01.513 DEBUG:teuthology.orchestra.run.smithi105:> fd = os.open("/home/ubuntu/cephtest/mnt.0/testdir", os.O_RDONLY) 2024-03-23T07:58:01.513 DEBUG:teuthology.orchestra.run.smithi105:> os.fchmod(fd, 0o777) 2024-03-23T07:58:01.513 DEBUG:teuthology.orchestra.run.smithi105:> os.fsync(fd) 2024-03-23T07:58:01.513 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-03-23T07:58:01.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:02.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.140+0000 7fd2cfe00640 1 Processor -- start 2024-03-23T07:58:02.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.140+0000 7fd2cfe00640 1 -- start start 2024-03-23T07:58:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2c80adbc0 0x7fd2c80b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80cbab0 0x7fd2c80b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cfe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2c806d390 0x7fd2c806d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cfe00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c806dd60 con 0x7fd2c806d390 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80b16a0 con 0x7fd2c80adbc0 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80b4700 con 0x7fd2c80cbab0 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cde00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2c806d390 0x7fd2c806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cde00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2c806d390 0x7fd2c806d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44136/0 (socket says 172.21.15.105:44136) 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cde00640 1 -- 172.21.15.105:0/2257880410 learned_addr learned my addr 172.21.15.105:0/2257880410 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cca00640 1 --2- 172.21.15.105:0/2257880410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80cbab0 0x7fd2c80b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.142+0000 7fd2cd400640 1 --2- 172.21.15.105:0/2257880410 >> v2:172.21.15.132:3300/0 conn(0x7fd2c80adbc0 0x7fd2c80b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.143+0000 7fd2cde00640 1 -- 172.21.15.105:0/2257880410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80cbab0 msgr2=0x7fd2c80b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.143+0000 7fd2cde00640 1 --2- 172.21.15.105:0/2257880410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80cbab0 0x7fd2c80b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.143+0000 7fd2cde00640 1 -- 172.21.15.105:0/2257880410 >> v2:172.21.15.132:3300/0 conn(0x7fd2c80adbc0 msgr2=0x7fd2c80b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:02.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.143+0000 7fd2cde00640 1 --2- 172.21.15.105:0/2257880410 >> v2:172.21.15.132:3300/0 conn(0x7fd2c80adbc0 0x7fd2c80b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.143+0000 7fd2cde00640 1 -- 172.21.15.105:0/2257880410 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80cbfb0 con 0x7fd2c806d390 2024-03-23T07:58:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.144+0000 7fd2cde00640 1 --2- 172.21.15.105:0/2257880410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c806d820 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd2c806e020 tx=0x7fd2c002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e04f95e3937ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.144+0000 7fd2c7e00640 1 -- 172.21.15.105:0/2257880410 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00317e0 con 0x7fd2c806d390 2024-03-23T07:58:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.144+0000 7fd2c7e00640 1 -- 172.21.15.105:0/2257880410 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c0031940 con 0x7fd2c806d390 2024-03-23T07:58:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.144+0000 7fd2c7e00640 1 -- 172.21.15.105:0/2257880410 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0031cd0 con 0x7fd2c806d390 2024-03-23T07:58:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2257880410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 msgr2=0x7fd2c806d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2257880410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c806d820 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd2c806e020 tx=0x7fd2c002b880 comp rx=0 tx=0).stop 2024-03-23T07:58:02.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2257880410 shutdown_connections 2024-03-23T07:58:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2257880410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c806d820 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2257880410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80cbab0 0x7fd2c80b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2257880410 >> v2:172.21.15.132:3300/0 conn(0x7fd2c80adbc0 0x7fd2c80b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2257880410 >> 172.21.15.105:0/2257880410 conn(0x7fd2c806a3d0 msgr2=0x7fd2c80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2257880410 shutdown_connections 2024-03-23T07:58:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.145+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2257880410 wait complete. 2024-03-23T07:58:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.147+0000 7fd2cfe00640 1 Processor -- start 2024-03-23T07:58:02.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.147+0000 7fd2cfe00640 1 -- start start 2024-03-23T07:58:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c8159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80adbc0 0x7fd2c813e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2c80cbab0 0x7fd2c8158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cfe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8139df0 con 0x7fd2c806d390 2024-03-23T07:58:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8139680 con 0x7fd2c80cbab0 2024-03-23T07:58:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c815a5c0 con 0x7fd2c80adbc0 2024-03-23T07:58:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c8159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c8159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44152/0 (socket says 172.21.15.105:44152) 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.148+0000 7fd2cd400640 1 -- 172.21.15.105:0/2404860640 learned_addr learned my addr 172.21.15.105:0/2404860640 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cde00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2c80cbab0 0x7fd2c8158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cca00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80adbc0 0x7fd2c813e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cd400640 1 -- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80adbc0 msgr2=0x7fd2c813e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cd400640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80adbc0 0x7fd2c813e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cd400640 1 -- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2c80cbab0 msgr2=0x7fd2c8158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cd400640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2c80cbab0 0x7fd2c8158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cd400640 1 -- 172.21.15.105:0/2404860640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c00094d0 con 0x7fd2c806d390 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cd400640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c8159f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009f20 tx=0x7fd2bc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2c5600640 1 -- 172.21.15.105:0/2404860640 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0155f0 con 0x7fd2c806d390 2024-03-23T07:58:02.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2c5600640 1 -- 172.21.15.105:0/2404860640 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc011040 con 0x7fd2c806d390 2024-03-23T07:58:02.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c815a8a0 con 0x7fd2c806d390 2024-03-23T07:58:02.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.149+0000 7fd2c5600640 1 -- 172.21.15.105:0/2404860640 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc015990 con 0x7fd2c806d390 2024-03-23T07:58:02.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.150+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c815ae60 con 0x7fd2c806d390 2024-03-23T07:58:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.151+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c806ac20 con 0x7fd2c806d390 2024-03-23T07:58:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.152+0000 7fd2c5600640 1 -- 172.21.15.105:0/2404860640 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd2bc015af0 con 0x7fd2c806d390 2024-03-23T07:58:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.152+0000 7fd2c5600640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd2a806ac30 0x7fd2a806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.153+0000 7fd2c5600640 1 -- 172.21.15.105:0/2404860640 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd2bc08bbb0 con 0x7fd2c806d390 2024-03-23T07:58:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.153+0000 7fd2cca00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd2a806ac30 0x7fd2a806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.153+0000 7fd2cca00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd2a806ac30 0x7fd2a806d0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd2c815b3d0 tx=0x7fd2b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:02.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.157+0000 7fd2c5600640 1 -- 172.21.15.105:0/2404860640 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd2bc056aa0 con 0x7fd2c806d390 2024-03-23T07:58:02.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.389+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c8073390 con 0x7fd2c806d390 2024-03-23T07:58:02.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.390+0000 7fd2c5600640 1 -- 172.21.15.105:0/2404860640 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fd2bc05a0c0 con 0x7fd2c806d390 2024-03-23T07:58:02.390 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:02.390 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:57:56.345100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/3723879365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3723879365},{"type":"v1","addr":"172.21.15.105:6837","nonce":3723879365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:02.390 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 16 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd2a806ac30 msgr2=0x7fd2a806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd2a806ac30 0x7fd2a806d0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd2c815b3d0 tx=0x7fd2b4008040 comp rx=0 tx=0).stop 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 msgr2=0x7fd2c8159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c8159f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009f20 tx=0x7fd2bc00f4a0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 shutdown_connections 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2c80cbab0 0x7fd2c8158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fd2a806ac30 0x7fd2a806d0f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2c80adbc0 0x7fd2c813e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 --2- 172.21.15.105:0/2404860640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c806d390 0x7fd2c8159f30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 >> 172.21.15.105:0/2404860640 conn(0x7fd2c806a3d0 msgr2=0x7fd2c81408f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:02.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.393+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 shutdown_connections 2024-03-23T07:58:02.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.394+0000 7fd2cfe00640 1 -- 172.21.15.105:0/2404860640 wait complete. 2024-03-23T07:58:02.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-23T07:58:02.650 INFO:teuthology.orchestra.run.smithi105.stdout:{"message":"","return_code":0} 2024-03-23T07:58:02.651 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-23T07:58:02.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.857+0000 7fdf96200640 1 Processor -- start 2024-03-23T07:58:02.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.857+0000 7fdf96200640 1 -- start start 2024-03-23T07:58:02.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.858+0000 7fdf96200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 0x7fdf9015b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.858+0000 7fdf96200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf900b39b0 0x7fdf900b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.859+0000 7fdf96200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9006c6f0 con 0x7fdf9015e500 2024-03-23T07:58:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.859+0000 7fdf96200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf900b4370 con 0x7fdf900b39b0 2024-03-23T07:58:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.859+0000 7fdf96200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9006c9d0 con 0x7fdf9015b560 2024-03-23T07:58:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.859+0000 7fdf8fe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdf9015e500 0x7fdf9015e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42390/0 (socket says 172.21.15.105:42390) 2024-03-23T07:58:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.859+0000 7fdf8ea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf900b39b0 0x7fdf900b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.859+0000 7fdf8fe00640 1 -- 172.21.15.105:0/510592263 learned_addr learned my addr 172.21.15.105:0/510592263 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.859+0000 7fdf8f400640 1 --2- 172.21.15.105:0/510592263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 0x7fdf9015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.860+0000 7fdf8ea00640 1 -- 172.21.15.105:0/510592263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 msgr2=0x7fdf9015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.860+0000 7fdf8ea00640 1 --2- 172.21.15.105:0/510592263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 0x7fdf9015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.860+0000 7fdf8ea00640 1 -- 172.21.15.105:0/510592263 >> v1:172.21.15.105:6789/0 conn(0x7fdf9015e500 legacy=0x7fdf9015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.860+0000 7fdf8ea00640 1 -- 172.21.15.105:0/510592263 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9006ccb0 con 0x7fdf900b39b0 2024-03-23T07:58:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.861+0000 7fdf8ea00640 1 --2- 172.21.15.105:0/510592263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900b3e30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097a0 tx=0x7fdf7c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b7f34446aac77f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.861+0000 7fdf8e000640 1 -- 172.21.15.105:0/510592263 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c02d040 con 0x7fdf900b39b0 2024-03-23T07:58:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.861+0000 7fdf8e000640 1 -- 172.21.15.105:0/510592263 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c031810 con 0x7fdf900b39b0 2024-03-23T07:58:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.861+0000 7fdf8e000640 1 -- 172.21.15.105:0/510592263 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c031ba0 con 0x7fdf900b39b0 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 -- 172.21.15.105:0/510592263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 msgr2=0x7fdf900b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 --2- 172.21.15.105:0/510592263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900b3e30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097a0 tx=0x7fdf7c02b8b0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 -- 172.21.15.105:0/510592263 shutdown_connections 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 --2- 172.21.15.105:0/510592263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900b3e30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 --2- 172.21.15.105:0/510592263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 0x7fdf9015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 -- 172.21.15.105:0/510592263 >> 172.21.15.105:0/510592263 conn(0x7fdf9006a3d0 msgr2=0x7fdf900b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 -- 172.21.15.105:0/510592263 shutdown_connections 2024-03-23T07:58:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.862+0000 7fdf96200640 1 -- 172.21.15.105:0/510592263 wait complete. 2024-03-23T07:58:02.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.864+0000 7fdf96200640 1 Processor -- start 2024-03-23T07:58:02.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.864+0000 7fdf96200640 1 -- start start 2024-03-23T07:58:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf96200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf96200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 0x7fdf901f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf96200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9015e500 0x7fdf90141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf96200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90151fa0 con 0x7fdf900b39b0 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf96200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901f3710 con 0x7fdf9015e500 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf96200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900a7e90 con 0x7fdf9015b560 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900a78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44178/0 (socket says 172.21.15.105:44178) 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 -- 172.21.15.105:0/4044131549 learned_addr learned my addr 172.21.15.105:0/4044131549 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 -- 172.21.15.105:0/4044131549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 msgr2=0x7fdf901f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 0x7fdf901f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 -- 172.21.15.105:0/4044131549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9015e500 msgr2=0x7fdf90141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9015e500 0x7fdf90141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.865+0000 7fdf8f400640 1 -- 172.21.15.105:0/4044131549 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c009400 con 0x7fdf900b39b0 2024-03-23T07:58:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.866+0000 7fdf8f400640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900a78b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000d800 tx=0x7fdf8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.866+0000 7fdf87400640 1 -- 172.21.15.105:0/4044131549 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000b6c0 con 0x7fdf900b39b0 2024-03-23T07:58:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.866+0000 7fdf87400640 1 -- 172.21.15.105:0/4044131549 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8000b820 con 0x7fdf900b39b0 2024-03-23T07:58:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.866+0000 7fdf87400640 1 -- 172.21.15.105:0/4044131549 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000bbb0 con 0x7fdf900b39b0 2024-03-23T07:58:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.866+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf900a8520 con 0x7fdf900b39b0 2024-03-23T07:58:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.866+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf90142430 con 0x7fdf900b39b0 2024-03-23T07:58:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.867+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9006ac20 con 0x7fdf900b39b0 2024-03-23T07:58:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.868+0000 7fdf87400640 1 -- 172.21.15.105:0/4044131549 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdf8000bd10 con 0x7fdf900b39b0 2024-03-23T07:58:02.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.869+0000 7fdf87400640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf7006ac00 0x7fdf7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:02.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.869+0000 7fdf87400640 1 -- 172.21.15.105:0/4044131549 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdf80089ac0 con 0x7fdf900b39b0 2024-03-23T07:58:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.869+0000 7fdf8ea00640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf7006ac00 0x7fdf7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.870+0000 7fdf8ea00640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf7006ac00 0x7fdf7006d0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c036000 tx=0x7fdf7c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:02.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:02.874+0000 7fdf87400640 1 -- 172.21.15.105:0/4044131549 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdf80054ae0 con 0x7fdf900b39b0 2024-03-23T07:58:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.107+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf90073390 con 0x7fdf900b39b0 2024-03-23T07:58:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.108+0000 7fdf87400640 1 -- 172.21.15.105:0/4044131549 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fdf80089e60 con 0x7fdf900b39b0 2024-03-23T07:58:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:03.108 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:57:56.345100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.105:6837/3723879365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3723879365},{"type":"v1","addr":"172.21.15.105:6837","nonce":3723879365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 16 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf7006ac00 msgr2=0x7fdf7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf7006ac00 0x7fdf7006d0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c036000 tx=0x7fdf7c030000 comp rx=0 tx=0).stop 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 msgr2=0x7fdf900a78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900a78b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000d800 tx=0x7fdf8000dcc0 comp rx=0 tx=0).stop 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 shutdown_connections 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9015e500 0x7fdf90141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf7006ac00 0x7fdf7006d0c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015b560 0x7fdf901f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 --2- 172.21.15.105:0/4044131549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf900b39b0 0x7fdf900a78b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 >> 172.21.15.105:0/4044131549 conn(0x7fdf9006a3d0 msgr2=0x7fdf900b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 shutdown_connections 2024-03-23T07:58:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:03.111+0000 7fdf96200640 1 -- 172.21.15.105:0/4044131549 wait complete. 2024-03-23T07:58:03.129 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_log_pause 1 2024-03-23T07:58:03.357 INFO:teuthology.orchestra.run.smithi105.stdout:{"success":"mds_log_pause = '' "} 2024-03-23T07:58:03.358 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_log_pause = '' " } 2024-03-23T07:58:03.358 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T07:58:03.358 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-03-23T07:58:03.358 DEBUG:teuthology.orchestra.run.smithi105:> os.chdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-03-23T07:58:03.358 DEBUG:teuthology.orchestra.run.smithi105:> os.setgid(65534) 2024-03-23T07:58:03.359 DEBUG:teuthology.orchestra.run.smithi105:> os.setuid(65534) 2024-03-23T07:58:03.359 DEBUG:teuthology.orchestra.run.smithi105:> fd = os.open("testfile", os.O_CREAT | os.O_RDWR, 0o644) 2024-03-23T07:58:03.359 DEBUG:teuthology.orchestra.run.smithi105:> os.fchmod(fd, 0o640) 2024-03-23T07:58:03.359 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-03-23T07:58:03.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:03.704 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod 600 /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-03-23T07:58:08.457 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:58:08.465 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:58:13.752 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 2:0 2024-03-23T07:58:13.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.973+0000 7f140be00640 1 Processor -- start 2024-03-23T07:58:13.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.973+0000 7f140be00640 1 -- start start 2024-03-23T07:58:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c15c710 0x7f140c15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140be00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f140c0cda00 0x7f140c070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140be00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c15d100 con 0x7f140c0cde60 2024-03-23T07:58:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140be00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c15d3e0 con 0x7f140c0cda00 2024-03-23T07:58:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c15d6c0 con 0x7f140c15c710 2024-03-23T07:58:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140a000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f140c0cda00 0x7f140c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140a000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f140c0cda00 0x7f140c070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43946/0 (socket says 172.21.15.105:43946) 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140a000640 1 -- 172.21.15.105:0/640595252 learned_addr learned my addr 172.21.15.105:0/640595252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140b400640 1 --1- 172.21.15.105:0/640595252 >> v1:172.21.15.105:6789/0 conn(0x7f140c0cde60 0x7f140c1604e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49438/0 (socket says 172.21.15.105:49438) 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.975+0000 7f140aa00640 1 --2- 172.21.15.105:0/640595252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c15c710 0x7f140c15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.976+0000 7f140a000640 1 -- 172.21.15.105:0/640595252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c15c710 msgr2=0x7f140c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.976+0000 7f140a000640 1 --2- 172.21.15.105:0/640595252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c15c710 0x7f140c15caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.976+0000 7f140a000640 1 -- 172.21.15.105:0/640595252 >> v1:172.21.15.105:6789/0 conn(0x7f140c0cde60 legacy=0x7f140c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.976+0000 7f140a000640 1 -- 172.21.15.105:0/640595252 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c15d9a0 con 0x7f140c0cda00 2024-03-23T07:58:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.976+0000 7f1409600640 1 -- 172.21.15.105:0/640595252 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808478600 0 0) 0x7f140c15d100 con 0x7f140c0cde60 2024-03-23T07:58:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.977+0000 7f140a000640 1 --2- 172.21.15.105:0/640595252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c0cda00 0x7f140c070020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65676c88bc70e937 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.977+0000 7f1409600640 1 -- 172.21.15.105:0/640595252 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8030840 con 0x7f140c0cda00 2024-03-23T07:58:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.977+0000 7f1409600640 1 -- 172.21.15.105:0/640595252 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80309a0 con 0x7f140c0cda00 2024-03-23T07:58:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.977+0000 7f1409600640 1 -- 172.21.15.105:0/640595252 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8030d30 con 0x7f140c0cda00 2024-03-23T07:58:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.977+0000 7f140be00640 1 -- 172.21.15.105:0/640595252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c0cda00 msgr2=0x7f140c070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.977+0000 7f140be00640 1 --2- 172.21.15.105:0/640595252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c0cda00 0x7f140c070020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802b980 comp rx=0 tx=0).stop 2024-03-23T07:58:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.978+0000 7f140be00640 1 -- 172.21.15.105:0/640595252 shutdown_connections 2024-03-23T07:58:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.978+0000 7f140be00640 1 --2- 172.21.15.105:0/640595252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c0cda00 0x7f140c070020 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.978+0000 7f140be00640 1 --2- 172.21.15.105:0/640595252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c15c710 0x7f140c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.978+0000 7f140be00640 1 -- 172.21.15.105:0/640595252 >> 172.21.15.105:0/640595252 conn(0x7f140c065e20 msgr2=0x7f140c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.978+0000 7f140be00640 1 -- 172.21.15.105:0/640595252 shutdown_connections 2024-03-23T07:58:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.978+0000 7f140be00640 1 -- 172.21.15.105:0/640595252 wait complete. 2024-03-23T07:58:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.979+0000 7f140be00640 1 Processor -- start 2024-03-23T07:58:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.980+0000 7f140be00640 1 -- start start 2024-03-23T07:58:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.980+0000 7f140be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c0cda00 0x7f140c1f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.980+0000 7f140be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f140c0cde60 0x7f140c14d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.980+0000 7f140be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c15c710 0x7f140c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.980+0000 7f140be00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0ac010 con 0x7f140c15c710 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.980+0000 7f140be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c15fab0 con 0x7f140c0cde60 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.980+0000 7f140be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c15fd90 con 0x7f140c0cda00 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c15c710 0x7f140c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c15c710 0x7f140c15f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43962/0 (socket says 172.21.15.105:43962) 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 -- 172.21.15.105:0/2637142745 learned_addr learned my addr 172.21.15.105:0/2637142745 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140aa00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c0cda00 0x7f140c1f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140a000640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f140c0cde60 0x7f140c14d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 -- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c0cda00 msgr2=0x7f140c1f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c0cda00 0x7f140c1f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 -- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f140c0cde60 msgr2=0x7f140c14d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f140c0cde60 0x7f140c14d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140b400640 1 -- 172.21.15.105:0/2637142745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80094d0 con 0x7f140c15c710 2024-03-23T07:58:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.981+0000 7f140aa00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c0cda00 0x7f140c1f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.982+0000 7f140b400640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c15c710 0x7f140c15f0d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1404001560 tx=0x7f140400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.982+0000 7f1402a00640 1 -- 172.21.15.105:0/2637142745 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400a640 con 0x7f140c15c710 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.982+0000 7f1402a00640 1 -- 172.21.15.105:0/2637142745 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140400a7a0 con 0x7f140c15c710 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.982+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c0ab950 con 0x7f140c15c710 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.982+0000 7f1402a00640 1 -- 172.21.15.105:0/2637142745 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400ab30 con 0x7f140c15c710 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.982+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c14d910 con 0x7f140c15c710 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.983+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c04e030 con 0x7f140c15c710 2024-03-23T07:58:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.984+0000 7f1402a00640 1 -- 172.21.15.105:0/2637142745 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f140400ac90 con 0x7f140c15c710 2024-03-23T07:58:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.985+0000 7f1402a00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f13ec06ac00 0x7f13ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.985+0000 7f1402a00640 1 -- 172.21.15.105:0/2637142745 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f1404088ad0 con 0x7f140c15c710 2024-03-23T07:58:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.985+0000 7f140aa00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f13ec06ac00 0x7f13ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.986+0000 7f140aa00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f13ec06ac00 0x7f13ec06d0c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f13fc006fd0 tx=0x7f13fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:13.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:13.990+0000 7f1402a00640 1 -- 172.21.15.105:0/2637142745 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1404053ae0 con 0x7f140c15c710 2024-03-23T07:58:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.252+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "2:0"} v 0) v1 -- 0x7f140c0b0110 con 0x7f140c15c710 2024-03-23T07:58:14.253 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-03-23T07:58:14.253+0000 7f31aa200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T07:58:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.273+0000 7f1402a00640 1 -- 172.21.15.105:0/2637142745 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "2:0"}]=0 failed mds gid 5262 v17) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1404018070 con 0x7f140c15c710 2024-03-23T07:58:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:failed mds gid 5262 2024-03-23T07:58:14.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.275+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f13ec06ac00 msgr2=0x7f13ec06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f13ec06ac00 0x7f13ec06d0c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f13fc006fd0 tx=0x7f13fc008040 comp rx=0 tx=0).stop 2024-03-23T07:58:14.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c15c710 msgr2=0x7f140c15f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c15c710 0x7f140c15f0d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1404001560 tx=0x7f140400cc70 comp rx=0 tx=0).stop 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 shutdown_connections 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140c15c710 0x7f140c15f0d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f140c0cde60 0x7f140c14d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f13ec06ac00 0x7f13ec06d0c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 --2- 172.21.15.105:0/2637142745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f140c0cda00 0x7f140c1f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 >> 172.21.15.105:0/2637142745 conn(0x7f140c065e20 msgr2=0x7f140c06c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 shutdown_connections 2024-03-23T07:58:14.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.276+0000 7f140be00640 1 -- 172.21.15.105:0/2637142745 wait complete. 2024-03-23T07:58:14.296 INFO:tasks.ceph.mds.a.smithi105.stderr: -2798> 2024-03-23T07:57:55.525+0000 7f9579000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T07:58:14.296 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:14.348 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-03-23T07:58:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.468+0000 7fc8f8000640 1 Processor -- start 2024-03-23T07:58:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.468+0000 7fc8f8000640 1 -- start start 2024-03-23T07:58:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.470+0000 7fc8f8000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc8f00b39b0 0x7fc8f00b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.470+0000 7fc8f8000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.470+0000 7fc8f8000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006c6f0 con 0x7fc8f015b560 2024-03-23T07:58:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.470+0000 7fc8f8000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f00b4370 con 0x7fc8f00b39b0 2024-03-23T07:58:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.470+0000 7fc8f8000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006c9d0 con 0x7fc8f015c400 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:58082/0 (socket says 172.21.15.105:58082) 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f6000640 1 -- 172.21.15.105:0/4114890597 learned_addr learned my addr 172.21.15.105:0/4114890597 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f4c00640 1 --2- 172.21.15.105:0/4114890597 >> v2:172.21.15.132:3300/0 conn(0x7fc8f00b39b0 0x7fc8f00b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f6000640 1 -- 172.21.15.105:0/4114890597 >> v2:172.21.15.132:3300/0 conn(0x7fc8f00b39b0 msgr2=0x7fc8f00b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f6000640 1 --2- 172.21.15.105:0/4114890597 >> v2:172.21.15.132:3300/0 conn(0x7fc8f00b39b0 0x7fc8f00b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f6000640 1 -- 172.21.15.105:0/4114890597 >> v1:172.21.15.132:6789/0 conn(0x7fc8f015b560 legacy=0x7fc8f015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.471+0000 7fc8f6000640 1 -- 172.21.15.105:0/4114890597 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f006ccb0 con 0x7fc8f015c400 2024-03-23T07:58:14.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.473+0000 7fc8f6000640 1 --2- 172.21.15.105:0/4114890597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f015c870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009e50 tx=0x7fc8e802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8602ff5bc58bf7f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.473+0000 7fc8efe00640 1 -- 172.21.15.105:0/4114890597 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e802cb00 con 0x7fc8f015c400 2024-03-23T07:58:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.473+0000 7fc8efe00640 1 -- 172.21.15.105:0/4114890597 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e802cc60 con 0x7fc8f015c400 2024-03-23T07:58:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.473+0000 7fc8efe00640 1 -- 172.21.15.105:0/4114890597 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80343f0 con 0x7fc8f015c400 2024-03-23T07:58:14.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 -- 172.21.15.105:0/4114890597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 msgr2=0x7fc8f015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 --2- 172.21.15.105:0/4114890597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f015c870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009e50 tx=0x7fc8e802bf60 comp rx=0 tx=0).stop 2024-03-23T07:58:14.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 -- 172.21.15.105:0/4114890597 shutdown_connections 2024-03-23T07:58:14.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 --2- 172.21.15.105:0/4114890597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f015c870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 --2- 172.21.15.105:0/4114890597 >> v2:172.21.15.132:3300/0 conn(0x7fc8f00b39b0 0x7fc8f00b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 -- 172.21.15.105:0/4114890597 >> 172.21.15.105:0/4114890597 conn(0x7fc8f006a3d0 msgr2=0x7fc8f00b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:14.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 -- 172.21.15.105:0/4114890597 shutdown_connections 2024-03-23T07:58:14.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.474+0000 7fc8f8000640 1 -- 172.21.15.105:0/4114890597 wait complete. 2024-03-23T07:58:14.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.476+0000 7fc8f8000640 1 Processor -- start 2024-03-23T07:58:14.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.476+0000 7fc8f8000640 1 -- start start 2024-03-23T07:58:14.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f8000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f00b39b0 0x7fc8f0154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:14.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f8000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8f015b560 0x7fc8f0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f8000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f013ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f8000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f0156f80 con 0x7fc8f00b39b0 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f8000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f0157260 con 0x7fc8f015b560 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f8000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f0157540 con 0x7fc8f015c400 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f00b39b0 0x7fc8f0154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f00b39b0 0x7fc8f0154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43986/0 (socket says 172.21.15.105:43986) 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 -- 172.21.15.105:0/545058141 learned_addr learned my addr 172.21.15.105:0/545058141 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 -- 172.21.15.105:0/545058141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 msgr2=0x7fc8f013ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f013ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 -- 172.21.15.105:0/545058141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8f015b560 msgr2=0x7fc8f0158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8f015b560 0x7fc8f0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.477+0000 7fc8f5600640 1 -- 172.21.15.105:0/545058141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e8009b30 con 0x7fc8f00b39b0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.478+0000 7fc8f5600640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f00b39b0 0x7fc8f0154610 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00c3c0 tx=0x7fc8dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.478+0000 7fc8ed600640 1 -- 172.21.15.105:0/545058141 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0052d0 con 0x7fc8f00b39b0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.478+0000 7fc8ed600640 1 -- 172.21.15.105:0/545058141 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc005450 con 0x7fc8f00b39b0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.478+0000 7fc8ed600640 1 -- 172.21.15.105:0/545058141 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc007260 con 0x7fc8f00b39b0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.478+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f0157820 con 0x7fc8f00b39b0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.478+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f0156c30 con 0x7fc8f00b39b0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.479+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f0157d40 con 0x7fc8f00b39b0 2024-03-23T07:58:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.480+0000 7fc8ed600640 1 -- 172.21.15.105:0/545058141 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc8dc0073c0 con 0x7fc8f00b39b0 2024-03-23T07:58:14.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.481+0000 7fc8ed600640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc8d006ac30 0x7fc8d006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:14.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.481+0000 7fc8ed600640 1 -- 172.21.15.105:0/545058141 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fc8dc0889f0 con 0x7fc8f00b39b0 2024-03-23T07:58:14.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.481+0000 7fc8f4c00640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc8d006ac30 0x7fc8d006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:14.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.482+0000 7fc8f4c00640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc8d006ac30 0x7fc8d006d0f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006fd0 tx=0x7fc8e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:14.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.486+0000 7fc8ed600640 1 -- 172.21.15.105:0/545058141 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc8f0157d40 con 0x7fc8f00b39b0 2024-03-23T07:58:14.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.715+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8f0157d40 con 0x7fc8f00b39b0 2024-03-23T07:58:14.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.716+0000 7fc8ed600640 1 -- 172.21.15.105:0/545058141 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3984 (secure 0 0 0) 0x7fc8f0157d40 con 0x7fc8f00b39b0 2024-03-23T07:58:14.716 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:14.716 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:58:14.273924+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5253":{"gid":5253,"name":"c","rank":0,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:14.716 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2024-03-23T07:58:14.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.718+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc8d006ac30 msgr2=0x7fc8d006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.718+0000 7fc8f8000640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc8d006ac30 0x7fc8d006d0f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006fd0 tx=0x7fc8e4008040 comp rx=0 tx=0).stop 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.718+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f00b39b0 msgr2=0x7fc8f0154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.718+0000 7fc8f8000640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f00b39b0 0x7fc8f0154610 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00c3c0 tx=0x7fc8dc00c880 comp rx=0 tx=0).stop 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 shutdown_connections 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc8d006ac30 0x7fc8d006d0f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8f015c400 0x7fc8f013ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8f015b560 0x7fc8f0158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 --2- 172.21.15.105:0/545058141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8f00b39b0 0x7fc8f0154610 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 >> 172.21.15.105:0/545058141 conn(0x7fc8f006a3d0 msgr2=0x7fc8f015d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 shutdown_connections 2024-03-23T07:58:14.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:14.719+0000 7fc8f8000640 1 -- 172.21.15.105:0/545058141 wait complete. 2024-03-23T07:58:15.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T07:57:53.325549+0000', 'modified': '2024-03-23T07:58:14.273924+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 44, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5253': {'gid': 5253, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/2670581804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 2670581804}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 2670581804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T07:58:15.140 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5253:up:replay 2024-03-23T07:58:16.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:16.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.363+0000 7f6271000640 1 Processor -- start 2024-03-23T07:58:16.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.363+0000 7f6271000640 1 -- start start 2024-03-23T07:58:16.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.364+0000 7f6271000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c0adbc0 0x7f626c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:16.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6271000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f626c0cbab0 0x7f626c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:16.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6271000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626c0b16a0 con 0x7f626c06d390 2024-03-23T07:58:16.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6271000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626c0b4700 con 0x7f626c0cbab0 2024-03-23T07:58:16.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6271000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c0cbfb0 con 0x7f626c0adbc0 2024-03-23T07:58:16.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6263e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f626c0cbab0 0x7f626c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:16.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6269e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c0adbc0 0x7f626c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:16.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6263e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f626c0cbab0 0x7f626c0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.105:52730/0 (socket says 172.21.15.105:52730) 2024-03-23T07:58:16.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.365+0000 7f6263e00640 1 -- 172.21.15.105:0/3526252846 learned_addr learned my addr 172.21.15.105:0/3526252846 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:16.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.366+0000 7f6263e00640 1 -- 172.21.15.105:0/3526252846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c0adbc0 msgr2=0x7f626c0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:16.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.366+0000 7f6263e00640 1 --2- 172.21.15.105:0/3526252846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c0adbc0 0x7f626c0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.366+0000 7f6263e00640 1 -- 172.21.15.105:0/3526252846 >> v1:172.21.15.132:6789/0 conn(0x7f626c06d390 legacy=0x7f626c0b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:16.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.366+0000 7f6263e00640 1 -- 172.21.15.105:0/3526252846 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0cc290 con 0x7f626c0cbab0 2024-03-23T07:58:16.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.368+0000 7f6263e00640 1 --2- 172.21.15.105:0/3526252846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0b20f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6254009870 tx=0x7f625402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb41db4e54455d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:16.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.368+0000 7f6269400640 1 -- 172.21.15.105:0/3526252846 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254031640 con 0x7f626c0cbab0 2024-03-23T07:58:16.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.368+0000 7f6269400640 1 -- 172.21.15.105:0/3526252846 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62540317a0 con 0x7f626c0cbab0 2024-03-23T07:58:16.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.368+0000 7f6269400640 1 -- 172.21.15.105:0/3526252846 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254031b30 con 0x7f626c0cbab0 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 -- 172.21.15.105:0/3526252846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 msgr2=0x7f626c0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 --2- 172.21.15.105:0/3526252846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0b20f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6254009870 tx=0x7f625402b8f0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 -- 172.21.15.105:0/3526252846 shutdown_connections 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 --2- 172.21.15.105:0/3526252846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0b20f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 --2- 172.21.15.105:0/3526252846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c0adbc0 0x7f626c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 -- 172.21.15.105:0/3526252846 >> 172.21.15.105:0/3526252846 conn(0x7f626c06a3d0 msgr2=0x7f626c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 -- 172.21.15.105:0/3526252846 shutdown_connections 2024-03-23T07:58:16.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.369+0000 7f6271000640 1 -- 172.21.15.105:0/3526252846 wait complete. 2024-03-23T07:58:16.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.371+0000 7f6271000640 1 Processor -- start 2024-03-23T07:58:16.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.371+0000 7f6271000640 1 -- start start 2024-03-23T07:58:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6271000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c06d390 0x7f626c158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6271000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f626c0adbc0 0x7f626c0ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6271000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6271000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c15cd80 con 0x7f626c0adbc0 2024-03-23T07:58:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6271000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c159210 con 0x7f626c0cbab0 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6271000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c13aa60 con 0x7f626c06d390 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6263e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f626c0adbc0 0x7f626c0ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6263e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f626c0adbc0 0x7f626c0ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43994/0 (socket says 172.21.15.105:43994) 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.372+0000 7f6263e00640 1 -- 172.21.15.105:0/3861476911 learned_addr learned my addr 172.21.15.105:0/3861476911 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f6269e00640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c06d390 0x7f626c158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f626a800640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f6263e00640 1 -- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c06d390 msgr2=0x7f626c158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f6263e00640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c06d390 0x7f626c158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f6263e00640 1 -- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 msgr2=0x7f626c0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f6263e00640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f6263e00640 1 -- 172.21.15.105:0/3861476911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540094d0 con 0x7f626c0adbc0 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f626a800640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.373+0000 7f6263e00640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f626c0adbc0 0x7f626c0ac9f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f62540090b0 tx=0x7f62540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.374+0000 7f6262000640 1 -- 172.21.15.105:0/3861476911 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625403b940 con 0x7f626c0adbc0 2024-03-23T07:58:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.374+0000 7f6262000640 1 -- 172.21.15.105:0/3861476911 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625403bac0 con 0x7f626c0adbc0 2024-03-23T07:58:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.374+0000 7f6262000640 1 -- 172.21.15.105:0/3861476911 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625403be70 con 0x7f626c0adbc0 2024-03-23T07:58:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.374+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c13ad40 con 0x7f626c0adbc0 2024-03-23T07:58:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.374+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c13b300 con 0x7f626c0adbc0 2024-03-23T07:58:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.375+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c0caef0 con 0x7f626c0adbc0 2024-03-23T07:58:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.376+0000 7f6262000640 1 -- 172.21.15.105:0/3861476911 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f625402f050 con 0x7f626c0adbc0 2024-03-23T07:58:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.377+0000 7f6262000640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f624806ac00 0x7f624806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.377+0000 7f6262000640 1 -- 172.21.15.105:0/3861476911 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f62540ab450 con 0x7f626c0adbc0 2024-03-23T07:58:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.377+0000 7f6269e00640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f624806ac00 0x7f624806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.378+0000 7f6269e00640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f624806ac00 0x7f624806d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f626c06d360 tx=0x7f625c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.381+0000 7f6262000640 1 -- 172.21.15.105:0/3861476911 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6254076390 con 0x7f626c0adbc0 2024-03-23T07:58:16.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.614+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f626c06ac20 con 0x7f626c0adbc0 2024-03-23T07:58:16.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.616+0000 7f6262000640 1 -- 172.21.15.105:0/3861476911 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f62540799b0 con 0x7f626c0adbc0 2024-03-23T07:58:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:16.615 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/12048571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":12048571},{"type":"v1","addr":"172.21.15.105:6837","nonce":12048571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:58:16.284753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5253":{"gid":5253,"name":"c","rank":0,"incarnation":18,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:16.616 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2024-03-23T07:58:16.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.618+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f624806ac00 msgr2=0x7f624806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:16.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.618+0000 7f6271000640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f624806ac00 0x7f624806d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f626c06d360 tx=0x7f625c008040 comp rx=0 tx=0).stop 2024-03-23T07:58:16.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.618+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f626c0adbc0 msgr2=0x7f626c0ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:16.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.618+0000 7f6271000640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f626c0adbc0 0x7f626c0ac9f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f62540090b0 tx=0x7f62540046c0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 shutdown_connections 2024-03-23T07:58:16.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f626c0cbab0 0x7f626c0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f626c0adbc0 0x7f626c0ac9f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f624806ac00 0x7f624806d0c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 --2- 172.21.15.105:0/3861476911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f626c06d390 0x7f626c158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:16.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 >> 172.21.15.105:0/3861476911 conn(0x7f626c06a3d0 msgr2=0x7f626c0b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:16.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 shutdown_connections 2024-03-23T07:58:16.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:16.619+0000 7f6271000640 1 -- 172.21.15.105:0/3861476911 wait complete. 2024-03-23T07:58:17.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T07:57:53.325549+0000', 'modified': '2024-03-23T07:58:16.284753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 44, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5253': {'gid': 5253, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.105:6835/2670581804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 2670581804}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 2670581804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T07:58:17.038 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5253:up:rejoin 2024-03-23T07:58:18.040 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:18.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.253+0000 7f97b4400640 1 Processor -- start 2024-03-23T07:58:18.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.253+0000 7f97b4400640 1 -- start start 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b4400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97ac0b37f0 0x7f97ac0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b4400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b4400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac06da70 con 0x7f97ac14aa40 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b4400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac06d7d0 con 0x7f97ac0b37f0 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b4400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b1d00 con 0x7f97ac0ccec0 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b1000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97ac0b37f0 0x7f97ac0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b1000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97ac0b37f0 0x7f97ac0cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54582/0 (socket says 172.21.15.105:54582) 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b1000640 1 -- 172.21.15.105:0/1602418526 learned_addr learned my addr 172.21.15.105:0/1602418526 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b1000640 1 -- 172.21.15.105:0/1602418526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 msgr2=0x7f97ac06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:18.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b1000640 1 --2- 172.21.15.105:0/1602418526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.255+0000 7f97b1000640 1 -- 172.21.15.105:0/1602418526 >> v1:172.21.15.132:6789/0 conn(0x7f97ac14aa40 legacy=0x7f97ac0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:18.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.256+0000 7f97b1000640 1 -- 172.21.15.105:0/1602418526 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0b1fe0 con 0x7f97ac0b37f0 2024-03-23T07:58:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.257+0000 7f97b1000640 1 --2- 172.21.15.105:0/1602418526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac0b37f0 0x7f97ac0cd7a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d76fb4846fcd776 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.257+0000 7f97abe00640 1 -- 172.21.15.105:0/1602418526 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798031640 con 0x7f97ac0b37f0 2024-03-23T07:58:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.257+0000 7f97abe00640 1 -- 172.21.15.105:0/1602418526 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97980317a0 con 0x7f97ac0b37f0 2024-03-23T07:58:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.257+0000 7f97abe00640 1 -- 172.21.15.105:0/1602418526 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798031b30 con 0x7f97ac0b37f0 2024-03-23T07:58:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 -- 172.21.15.105:0/1602418526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac0b37f0 msgr2=0x7f97ac0cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 --2- 172.21.15.105:0/1602418526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac0b37f0 0x7f97ac0cd7a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979802b8f0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 -- 172.21.15.105:0/1602418526 shutdown_connections 2024-03-23T07:58:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 --2- 172.21.15.105:0/1602418526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 --2- 172.21.15.105:0/1602418526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac0b37f0 0x7f97ac0cd7a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 -- 172.21.15.105:0/1602418526 >> 172.21.15.105:0/1602418526 conn(0x7f97ac06a3d0 msgr2=0x7f97ac0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 -- 172.21.15.105:0/1602418526 shutdown_connections 2024-03-23T07:58:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.258+0000 7f97b4400640 1 -- 172.21.15.105:0/1602418526 wait complete. 2024-03-23T07:58:18.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.259+0000 7f97b4400640 1 Processor -- start 2024-03-23T07:58:18.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.260+0000 7f97b4400640 1 -- start start 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.260+0000 7f97b4400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ac0b37f0 0x7f97ac160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.260+0000 7f97b4400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac13e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.260+0000 7f97b4400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac14aa40 0x7f97ac13f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.260+0000 7f97b4400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac13f9b0 con 0x7f97ac14aa40 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.260+0000 7f97b4400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac13eec0 con 0x7f97ac0b37f0 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b4400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac145b50 con 0x7f97ac0ccec0 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac14aa40 0x7f97ac13f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac14aa40 0x7f97ac13f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54584/0 (socket says 172.21.15.105:54584) 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 -- 172.21.15.105:0/634453428 learned_addr learned my addr 172.21.15.105:0/634453428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b1a00640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ac0b37f0 0x7f97ac160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b1000640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac13e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 -- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 msgr2=0x7f97ac13e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac13e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 -- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ac0b37f0 msgr2=0x7f97ac160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ac0b37f0 0x7f97ac160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b2400640 1 -- 172.21.15.105:0/634453428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980094d0 con 0x7f97ac14aa40 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.261+0000 7f97b1000640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac13e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:18.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.262+0000 7f97b2400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac14aa40 0x7f97ac13f240 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f97a400b9a0 tx=0x7f97a400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.262+0000 7f97a9600640 1 -- 172.21.15.105:0/634453428 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400cba0 con 0x7f97ac14aa40 2024-03-23T07:58:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.262+0000 7f97a9600640 1 -- 172.21.15.105:0/634453428 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400cd20 con 0x7f97ac14aa40 2024-03-23T07:58:18.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.262+0000 7f97a9600640 1 -- 172.21.15.105:0/634453428 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4014680 con 0x7f97ac14aa40 2024-03-23T07:58:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.262+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac13f780 con 0x7f97ac14aa40 2024-03-23T07:58:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.262+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1421d0 con 0x7f97ac14aa40 2024-03-23T07:58:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.263+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac143160 con 0x7f97ac14aa40 2024-03-23T07:58:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.264+0000 7f97a9600640 1 -- 172.21.15.105:0/634453428 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f97a401d3b0 con 0x7f97ac14aa40 2024-03-23T07:58:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.265+0000 7f97a9600640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f978c06ac00 0x7f978c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.265+0000 7f97a9600640 1 -- 172.21.15.105:0/634453428 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f97a4089590 con 0x7f97ac14aa40 2024-03-23T07:58:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.265+0000 7f97b1a00640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f978c06ac00 0x7f978c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.266+0000 7f97b1a00640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f978c06ac00 0x7f978c06d0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006fd0 tx=0x7f97a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:18.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.270+0000 7f97a9600640 1 -- 172.21.15.105:0/634453428 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f97a40544c0 con 0x7f97ac14aa40 2024-03-23T07:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.508+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97ac073390 con 0x7f97ac14aa40 2024-03-23T07:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.509+0000 7f97a9600640 1 -- 172.21.15.105:0/634453428 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f97a4057ae0 con 0x7f97ac14aa40 2024-03-23T07:58:18.509 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:18.509 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/12048571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":12048571},{"type":"v1","addr":"172.21.15.105:6837","nonce":12048571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:58:18.306711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5253":{"gid":5253,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 22 2024-03-23T07:58:18.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f978c06ac00 msgr2=0x7f978c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f978c06ac00 0x7f978c06d0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006fd0 tx=0x7f97a0008040 comp rx=0 tx=0).stop 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac14aa40 msgr2=0x7f97ac13f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac14aa40 0x7f97ac13f240 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f97a400b9a0 tx=0x7f97a400be60 comp rx=0 tx=0).stop 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 shutdown_connections 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ac14aa40 0x7f97ac13f240 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f978c06ac00 0x7f978c06d0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ac0ccec0 0x7f97ac13e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 --2- 172.21.15.105:0/634453428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ac0b37f0 0x7f97ac160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 >> 172.21.15.105:0/634453428 conn(0x7f97ac06a3d0 msgr2=0x7f97ac0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:18.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 shutdown_connections 2024-03-23T07:58:18.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:18.512+0000 7f97b4400640 1 -- 172.21.15.105:0/634453428 wait complete. 2024-03-23T07:58:18.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T07:57:53.325549+0000', 'modified': '2024-03-23T07:58:18.306711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 44, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5253': {'gid': 5253, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.105:6835/2670581804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 2670581804}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 2670581804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T07:58:18.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T07:58:18.934 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T07:58:19.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.154+0000 7f821ce00640 1 Processor -- start 2024-03-23T07:58:19.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.154+0000 7f821ce00640 1 -- start start 2024-03-23T07:58:19.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f821ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f82180cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f821ce00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8218160e90 0x7f82180d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f821ce00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82180ccd90 con 0x7f821815c820 2024-03-23T07:58:19.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f821ce00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82180cdd70 con 0x7f8218160e90 2024-03-23T07:58:19.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f821ce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82180b7fc0 con 0x7f82180cf7c0 2024-03-23T07:58:19.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f8217400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f821815c820 0x7f82180cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49360/0 (socket says 172.21.15.105:49360) 2024-03-23T07:58:19.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f8217400640 1 -- 172.21.15.105:0/418748613 learned_addr learned my addr 172.21.15.105:0/418748613 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:19.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.156+0000 7f8217e00640 1 --2- 172.21.15.105:0/418748613 >> v2:172.21.15.105:3300/0 conn(0x7f8218160e90 0x7f82180d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.157+0000 7f820ea00640 1 --2- 172.21.15.105:0/418748613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f82180cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.157+0000 7f8216a00640 1 -- 172.21.15.105:0/418748613 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831075540 0 0) 0x7f82180ccd90 con 0x7f821815c820 2024-03-23T07:58:19.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.157+0000 7f8216a00640 1 -- 172.21.15.105:0/418748613 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81fc003520 con 0x7f821815c820 2024-03-23T07:58:19.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.157+0000 7f8217e00640 1 -- 172.21.15.105:0/418748613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 msgr2=0x7f82180cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.157+0000 7f8217e00640 1 --2- 172.21.15.105:0/418748613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f82180cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.157+0000 7f8217e00640 1 -- 172.21.15.105:0/418748613 >> v1:172.21.15.105:6789/0 conn(0x7f821815c820 legacy=0x7f82180cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.157+0000 7f8217e00640 1 -- 172.21.15.105:0/418748613 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82180b82a0 con 0x7f8218160e90 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.158+0000 7f8217e00640 1 --2- 172.21.15.105:0/418748613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180d0560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8210009d90 tx=0x7f821002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a96b163ec8e4af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.158+0000 7f8216a00640 1 -- 172.21.15.105:0/418748613 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821002cce0 con 0x7f8218160e90 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.158+0000 7f8216a00640 1 -- 172.21.15.105:0/418748613 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821002ce40 con 0x7f8218160e90 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.158+0000 7f8216a00640 1 -- 172.21.15.105:0/418748613 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210034660 con 0x7f8218160e90 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 -- 172.21.15.105:0/418748613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 msgr2=0x7f82180d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 --2- 172.21.15.105:0/418748613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180d0560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8210009d90 tx=0x7f821002bf30 comp rx=0 tx=0).stop 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 -- 172.21.15.105:0/418748613 shutdown_connections 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 --2- 172.21.15.105:0/418748613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180d0560 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 --2- 172.21.15.105:0/418748613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f82180cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 -- 172.21.15.105:0/418748613 >> 172.21.15.105:0/418748613 conn(0x7f821806b070 msgr2=0x7f821806b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:19.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 -- 172.21.15.105:0/418748613 shutdown_connections 2024-03-23T07:58:19.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.159+0000 7f821ce00640 1 -- 172.21.15.105:0/418748613 wait complete. 2024-03-23T07:58:19.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.161+0000 7f821ce00640 1 Processor -- start 2024-03-23T07:58:19.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.161+0000 7f821ce00640 1 -- start start 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f821ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f8218157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f821ce00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821815c820 0x7f82180cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f821ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f821ce00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821815f890 con 0x7f8218160e90 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f821ce00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821815fb70 con 0x7f821815c820 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f821ce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821815fe50 con 0x7f82180cf7c0 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f8217e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f8217e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54602/0 (socket says 172.21.15.105:54602) 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.162+0000 7f8217e00640 1 -- 172.21.15.105:0/1812777668 learned_addr learned my addr 172.21.15.105:0/1812777668 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f8217400640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f8218157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f820ea00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821815c820 0x7f82180cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f8217e00640 1 -- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 msgr2=0x7f8218157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f8217e00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f8218157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f8217e00640 1 -- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821815c820 msgr2=0x7f82180cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f8217e00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821815c820 0x7f82180cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f8217e00640 1 -- 172.21.15.105:0/1812777668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210009a70 con 0x7f8218160e90 2024-03-23T07:58:19.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f8217e00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180cb610 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8210009d90 tx=0x7f8210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f820fe00640 1 -- 172.21.15.105:0/1812777668 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821003d600 con 0x7f8218160e90 2024-03-23T07:58:19.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f820fe00640 1 -- 172.21.15.105:0/1812777668 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821003d780 con 0x7f8218160e90 2024-03-23T07:58:19.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f820fe00640 1 -- 172.21.15.105:0/1812777668 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821003db30 con 0x7f8218160e90 2024-03-23T07:58:19.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.163+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8218160130 con 0x7f8218160e90 2024-03-23T07:58:19.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.164+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8218158020 con 0x7f8218160e90 2024-03-23T07:58:19.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.166+0000 7f820fe00640 1 -- 172.21.15.105:0/1812777668 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f821003b040 con 0x7f8218160e90 2024-03-23T07:58:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.167+0000 7f820fe00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f81f406ac00 0x7f81f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.167+0000 7f820fe00640 1 -- 172.21.15.105:0/1812777668 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f82100abad0 con 0x7f8218160e90 2024-03-23T07:58:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.168+0000 7f8217400640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f81f406ac00 0x7f81f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.168+0000 7f82211cc740 1 Processor -- start 2024-03-23T07:58:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.168+0000 7f82211cc740 1 -- start start 2024-03-23T07:58:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.168+0000 7f8217400640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f81f406ac00 0x7f81f406d0c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82080011e0 tx=0x7f8208009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee42e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f82211cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee72c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee43390 0x55cd5ee73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f82211cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ee1e2c0 con 0x55cd5ee70290 2024-03-23T07:58:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5edb38f0 con 0x55cd5ee6feb0 2024-03-23T07:58:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5edb1080 con 0x55cd5ee43390 2024-03-23T07:58:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee72c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee72c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54616/0 (socket says 172.21.15.105:54616) 2024-03-23T07:58:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 -- 172.21.15.105:0/2758413910 learned_addr learned my addr 172.21.15.105:0/2758413910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 -- 172.21.15.105:0/2758413910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee43390 msgr2=0x55cd5ee73180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 --2- 172.21.15.105:0/2758413910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee43390 0x55cd5ee73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 -- 172.21.15.105:0/2758413910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 msgr2=0x55cd5ee42e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 --2- 172.21.15.105:0/2758413910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee42e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.169+0000 7f820ea00640 1 -- 172.21.15.105:0/2758413910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5ed77a10 con 0x55cd5ee70290 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.170+0000 7f820ea00640 1 --2- 172.21.15.105:0/2758413910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee72c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f820000b7b0 tx=0x7f820000bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919ad6aed29ffc7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.170+0000 7f8207e00640 1 -- 172.21.15.105:0/2758413910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200004020 con 0x55cd5ee70290 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.170+0000 7f8207e00640 1 -- 172.21.15.105:0/2758413910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200004180 con 0x55cd5ee70290 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.170+0000 7f8207e00640 1 -- 172.21.15.105:0/2758413910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000f7c0 con 0x55cd5ee70290 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 -- 172.21.15.105:0/2758413910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 msgr2=0x55cd5ee72c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 --2- 172.21.15.105:0/2758413910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee72c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f820000b7b0 tx=0x7f820000bc70 comp rx=0 tx=0).stop 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 -- 172.21.15.105:0/2758413910 shutdown_connections 2024-03-23T07:58:19.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 --2- 172.21.15.105:0/2758413910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee43390 0x55cd5ee73180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 --2- 172.21.15.105:0/2758413910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee72c40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 --2- 172.21.15.105:0/2758413910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee42e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 -- 172.21.15.105:0/2758413910 >> 172.21.15.105:0/2758413910 conn(0x55cd5ee450c0 msgr2=0x55cd5ee8dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 -- 172.21.15.105:0/2758413910 shutdown_connections 2024-03-23T07:58:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.171+0000 7f82211cc740 1 -- 172.21.15.105:0/2758413910 wait complete. 2024-03-23T07:58:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.172+0000 7f82211cc740 1 -- start start 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.172+0000 7f82211cc740 1 Processor -- start 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f82211cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 0x55cd5ee60410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5ee60950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f82211cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5edac970 con 0x55cd5ee6feb0 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ed4e0a0 con 0x55cd5ee70290 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ed4dd40 con 0x55cd5ee5ddf0 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f82211cc740 10 client.0 _ll_register_callbacks cb 0x55cd5eda5fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee852f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54624/0 (socket says 172.21.15.105:54624) 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 -- 172.21.15.105:0/2031509705 learned_addr learned my addr 172.21.15.105:0/2031509705 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f820ea00640 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 0x55cd5ee60410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217e00640 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5ee60950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 -- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 msgr2=0x55cd5ee60950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5ee60950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 -- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 msgr2=0x55cd5ee60410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 0x55cd5ee60410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.173+0000 7f8217400640 1 -- 172.21.15.105:0/2031509705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820000e040 con 0x55cd5ee6feb0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f8217400640 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee852f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8208005e60 tx=0x7f8208001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67584ea70fc7e386 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f8204c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f8204c00640 1 -- 172.21.15.105:0/2031509705 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208007770 con 0x55cd5ee6feb0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f8204c00640 1 -- 172.21.15.105:0/2031509705 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8208023ce0 con 0x55cd5ee6feb0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd5edd6b10 con 0x55cd5ee6feb0 2024-03-23T07:58:19.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f8204c00640 1 -- 172.21.15.105:0/2031509705 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8208023ea0 con 0x55cd5ee6feb0 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f82211cc740 10 client.5514 fetch_fsmap learned FSMap version 22 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.174+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55cd5edaaff0 con 0x55cd5ee6feb0 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f8204c00640 1 -- 172.21.15.105:0/2031509705 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd5edd6b10 con 0x55cd5ee6feb0 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f8204c00640 1 -- 172.21.15.105:0/2031509705 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 22) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f820800ecb0 con 0x55cd5ee6feb0 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f82211cc740 10 client.5514 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f82211cc740 10 client.5514 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f82211cc740 20 client.5514 populate_metadata read hostname 'smithi105' 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f82211cc740 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee5f260 0x55cd5ee5f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f82211cc740 4 client.5514 mds_command: new command op to 5253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 --> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5eaa03c0 con 0x55cd5ee5f260 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.175+0000 7f820ea00640 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee5f260 0x55cd5ee5f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.176+0000 7f8204c00640 10 client.5514 ms_handle_connect on v2:172.21.15.105:6834/2670581804 2024-03-23T07:58:19.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.176+0000 7f820ea00640 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee5f260 0x55cd5ee5f640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1e7c4e9a6e7576 server_cookie=4a077a8739ddd28e in_seq=0 out_seq=0 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.176+0000 7f8204c00640 1 -- 172.21.15.105:0/2031509705 <== mds.0 v2:172.21.15.105:6834/2670581804 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cd5eaa03c0 con 0x55cd5ee5f260 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.176+0000 7f8204c00640 10 client.5514 handle_command_reply: tid=0 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.176+0000 7f82211cc740 1 client.5514 shutdown 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee6feb0 msgr2=0x55cd5ee852f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee852f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8208005e60 tx=0x7f8208001e70 comp rx=0 tx=0).stop 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 shutdown_connections 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee5f260 0x55cd5ee5f640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5ee60950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 0x55cd5ee60410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 --2- 172.21.15.105:0/2031509705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee852f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f8204c00640 0 client.5514 ms_handle_reset on v2:172.21.15.105:6834/2670581804 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 >> 172.21.15.105:0/2031509705 conn(0x55cd5ee43390 msgr2=0x55cd5ee42530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 shutdown_connections 2024-03-23T07:58:19.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 1 -- 172.21.15.105:0/2031509705 wait complete. 2024-03-23T07:58:19.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.177+0000 7f82211cc740 20 client.5514 trim_cache size 0 max 16384 2024-03-23T07:58:19.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.193+0000 7f82211cc740 1 Processor -- start 2024-03-23T07:58:19.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.193+0000 7f82211cc740 1 -- start start 2024-03-23T07:58:19.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.194+0000 7f82211cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee5ddf0 0x55cd5ee70670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.194+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee6feb0 0x55cd5ee84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.194+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 0x55cd5ee76480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.194+0000 7f82211cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5eb13f20 con 0x55cd5ee5ddf0 2024-03-23T07:58:19.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.194+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ee64930 con 0x55cd5ee70290 2024-03-23T07:58:19.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.194+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ee84e80 con 0x55cd5ee6feb0 2024-03-23T07:58:19.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee5ddf0 0x55cd5ee70670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee5ddf0 0x55cd5ee70670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54638/0 (socket says 172.21.15.105:54638) 2024-03-23T07:58:19.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 -- 172.21.15.105:0/1358946310 learned_addr learned my addr 172.21.15.105:0/1358946310 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:19.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 -- 172.21.15.105:0/1358946310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee6feb0 msgr2=0x55cd5ee84790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 --2- 172.21.15.105:0/1358946310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee6feb0 0x55cd5ee84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 -- 172.21.15.105:0/1358946310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 msgr2=0x55cd5ee76480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 --2- 172.21.15.105:0/1358946310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 0x55cd5ee76480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 -- 172.21.15.105:0/1358946310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5ede07c0 con 0x55cd5ee5ddf0 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.195+0000 7f8217400640 1 --2- 172.21.15.105:0/1358946310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee5ddf0 0x55cd5ee70670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8208000f10 tx=0x7f820801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8678d4b733c97760 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f820cc00640 1 -- 172.21.15.105:0/1358946310 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820800ed80 con 0x55cd5ee5ddf0 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f820cc00640 1 -- 172.21.15.105:0/1358946310 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208019040 con 0x55cd5ee5ddf0 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f820cc00640 1 -- 172.21.15.105:0/1358946310 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208013b10 con 0x55cd5ee5ddf0 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 -- 172.21.15.105:0/1358946310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee5ddf0 msgr2=0x55cd5ee70670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 --2- 172.21.15.105:0/1358946310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee5ddf0 0x55cd5ee70670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8208000f10 tx=0x7f820801d000 comp rx=0 tx=0).stop 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 -- 172.21.15.105:0/1358946310 shutdown_connections 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 --2- 172.21.15.105:0/1358946310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee70290 0x55cd5ee76480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 --2- 172.21.15.105:0/1358946310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee6feb0 0x55cd5ee84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 --2- 172.21.15.105:0/1358946310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee5ddf0 0x55cd5ee70670 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 -- 172.21.15.105:0/1358946310 >> 172.21.15.105:0/1358946310 conn(0x55cd5ee43390 msgr2=0x55cd5ee5cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:19.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.196+0000 7f82211cc740 1 -- 172.21.15.105:0/1358946310 shutdown_connections 2024-03-23T07:58:19.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.197+0000 7f82211cc740 1 -- 172.21.15.105:0/1358946310 wait complete. 2024-03-23T07:58:19.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.197+0000 7f82211cc740 1 -- start start 2024-03-23T07:58:19.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.197+0000 7f82211cc740 1 Processor -- start 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.198+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5eef3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.198+0000 7f82211cc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee84150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.198+0000 7f82211cc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.198+0000 7f82211cc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ee6eb10 con 0x55cd5ee70290 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.198+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ee63e40 con 0x55cd5ee6feb0 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.198+0000 7f82211cc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ee63680 con 0x55cd5ee5ddf0 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.198+0000 7f82211cc740 10 client.0 _ll_register_callbacks cb 0x7f8208001430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee91150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54652/0 (socket says 172.21.15.105:54652) 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 -- 172.21.15.105:0/3940260972 learned_addr learned my addr 172.21.15.105:0/3940260972 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217400640 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5eef3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f820ea00640 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee84150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 -- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 msgr2=0x55cd5eef3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5eef3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 -- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 msgr2=0x55cd5ee84150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee84150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 -- 172.21.15.105:0/3940260972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5edaaff0 con 0x55cd5ee70290 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8217e00640 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee91150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f821000c070 tx=0x7f821007a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ed3d8bbf137812 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8204c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.199+0000 7f8204c00640 1 -- 172.21.15.105:0/3940260972 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821007e6b0 con 0x55cd5ee70290 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f8204c00640 1 -- 172.21.15.105:0/3940260972 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8210082c70 con 0x55cd5ee70290 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8200004180 con 0x55cd5ee70290 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f8204c00640 1 -- 172.21.15.105:0/3940260972 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f821003dc90 con 0x55cd5ee70290 2024-03-23T07:58:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 10 client.5520 fetch_fsmap learned FSMap version 22 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55cd5ed4d2b0 con 0x55cd5ee70290 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f8204c00640 1 -- 172.21.15.105:0/3940260972 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200004180 con 0x55cd5ee70290 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f8204c00640 1 -- 172.21.15.105:0/3940260972 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 22) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8210079d70 con 0x55cd5ee70290 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 10 client.5520 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 10 client.5520 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 20 client.5520 populate_metadata read hostname 'smithi105' 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee73a70 0x55cd5ee73e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 4 client.5520 mds_command: new command op to 5253 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 --> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cd5eab16c0 con 0x55cd5ee73a70 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.200+0000 7f8217400640 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee73a70 0x55cd5ee73e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f8217400640 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee73a70 0x55cd5ee73e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1246b5f8f78f5960 server_cookie=f689c0a9416c3c85 in_seq=0 out_seq=0 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f8204c00640 10 client.5520 ms_handle_connect on v2:172.21.15.105:6834/2670581804 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f8204c00640 1 -- 172.21.15.105:0/3940260972 <== mds.0 v2:172.21.15.105:6834/2670581804 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55cd5eab16c0 con 0x55cd5ee73a70 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f8204c00640 10 client.5520 handle_command_reply: tid=0 2024-03-23T07:58:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 client.5520 shutdown 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 msgr2=0x55cd5ee91150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee91150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f821000c070 tx=0x7f821007a4f0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 shutdown_connections 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cd5ee70290 0x55cd5ee91150 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cd5ee6feb0 0x55cd5ee84150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.105:6834/2670581804,v1:172.21.15.105:6835/2670581804] conn(0x55cd5ee73a70 0x55cd5ee73e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 --2- 172.21.15.105:0/3940260972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cd5ee5ddf0 0x55cd5eef3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 >> 172.21.15.105:0/3940260972 conn(0x55cd5ee43390 msgr2=0x55cd5ee884c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.201+0000 7f8204c00640 0 client.5520 ms_handle_reset on v2:172.21.15.105:6834/2670581804 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 shutdown_connections 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f82211cc740 1 -- 172.21.15.105:0/3940260972 wait complete. 2024-03-23T07:58:19.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f82211cc740 20 client.5520 trim_cache size 0 max 16384 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "e641b836-a0e5-4e95-93aa-bb0628868335", 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5253, 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 4.9203128969999996, 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 22, 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 44, 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 44, 2024-03-23T07:58:19.206 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 34.416622943999997 2024-03-23T07:58:19.207 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:19.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f81f406ac00 msgr2=0x7f81f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f81f406ac00 0x7f81f406d0c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82080011e0 tx=0x7f8208009040 comp rx=0 tx=0).stop 2024-03-23T07:58:19.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 msgr2=0x7f82180cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180cb610 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8210009d90 tx=0x7f8210004a40 comp rx=0 tx=0).stop 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 shutdown_connections 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218160e90 0x7f82180cb610 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821815c820 0x7f82180cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f81f406ac00 0x7f81f406d0c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 --2- 172.21.15.105:0/1812777668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82180cf7c0 0x7f8218157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.202+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 >> 172.21.15.105:0/1812777668 conn(0x7f821806b070 msgr2=0x7f82180c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.203+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 shutdown_connections 2024-03-23T07:58:19.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.203+0000 7f821ce00640 1 -- 172.21.15.105:0/1812777668 wait complete. 2024-03-23T07:58:19.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:19.218 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%a /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-03-23T07:58:19.266 INFO:teuthology.orchestra.run.smithi105.stdout:600 2024-03-23T07:58:19.268 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T07:58:19.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T07:58:19.372 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T07:58:19.491 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T07:58:19.491 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:58:19.491 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:19.548 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:58:19.548 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:19.600 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:58:19.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.813+0000 7f92f5e00640 1 Processor -- start 2024-03-23T07:58:19.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.814+0000 7f92f5e00640 1 -- start start 2024-03-23T07:58:19.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 0x7f92f00cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92f5e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f92f00b4350 0x7f92f00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92f5e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f015d0c0 con 0x7f92f00b1740 2024-03-23T07:58:19.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92f5e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f00b4d20 con 0x7f92f00b4350 2024-03-23T07:58:19.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92f5e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00d0ae0 con 0x7f92f015cce0 2024-03-23T07:58:19.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92ee000640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f92f00b1740 0x7f92f00b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49362/0 (socket says 172.21.15.105:49362) 2024-03-23T07:58:19.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92ee000640 1 -- 172.21.15.105:0/2709438004 learned_addr learned my addr 172.21.15.105:0/2709438004 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.815+0000 7f92eea00640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 0x7f92f00cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ef400640 1 --2- 172.21.15.105:0/2709438004 >> v2:172.21.15.132:3300/0 conn(0x7f92f00b4350 0x7f92f00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392552529 0 0) 0x7f92f015d0c0 con 0x7f92f00b1740 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d4003520 con 0x7f92f00b1740 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 107416293 0 0) 0x7f92d4003520 con 0x7f92f00b1740 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 msgr2=0x7f92f00cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 0x7f92f00cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 >> v2:172.21.15.132:3300/0 conn(0x7f92f00b4350 msgr2=0x7f92f00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 --2- 172.21.15.105:0/2709438004 >> v2:172.21.15.132:3300/0 conn(0x7f92f00b4350 0x7f92f00b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00d0d80 con 0x7f92f00b1740 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f92d8001670 con 0x7f92f00b1740 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 >> v1:172.21.15.105:6789/0 conn(0x7f92f00b1740 legacy=0x7f92f00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 0x7f92d4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92d4003ef0 0x7f92d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4004d00 0x7f92d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00b4d20 con 0x7f92d4003ef0 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00d0ae0 con 0x7f92d4004d00 2024-03-23T07:58:19.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400b980 con 0x7f92f015cce0 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.816+0000 7f92ee000640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92d4003ef0 0x7f92d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.817+0000 7f92ef400640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4004d00 0x7f92d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.817+0000 7f92eea00640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 0x7f92d4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.817+0000 7f92ee000640 1 -- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 msgr2=0x7f92d4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.817+0000 7f92ee000640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f015cce0 0x7f92d4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.817+0000 7f92ee000640 1 -- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4004d00 msgr2=0x7f92d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.817+0000 7f92ee000640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4004d00 0x7f92d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.817+0000 7f92ee000640 1 -- 172.21.15.105:0/2709438004 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4003520 con 0x7f92d4003ef0 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.818+0000 7f92ee000640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92d4003ef0 0x7f92d40047c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c240 tx=0x7f92d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1975fc72bfb048 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.818+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d802cae0 con 0x7f92d4003ef0 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.818+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d802cc40 con 0x7f92d4003ef0 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.818+0000 7f92ed600640 1 -- 172.21.15.105:0/2709438004 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8005af0 con 0x7f92d4003ef0 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.819+0000 7f92f5e00640 1 -- 172.21.15.105:0/2709438004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92d4003ef0 msgr2=0x7f92d40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.819+0000 7f92f5e00640 1 --2- 172.21.15.105:0/2709438004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92d4003ef0 0x7f92d40047c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c240 tx=0x7f92d40057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.819+0000 7f92ef400640 1 -- 172.21.15.105:0/2709438004 reap_dead start 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.819+0000 7f92f5e00640 1 -- 172.21.15.105:0/2709438004 shutdown_connections 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.819+0000 7f92f5e00640 1 -- 172.21.15.105:0/2709438004 >> 172.21.15.105:0/2709438004 conn(0x7f92f00cf350 msgr2=0x7f92f00cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.819+0000 7f92f5e00640 1 -- 172.21.15.105:0/2709438004 shutdown_connections 2024-03-23T07:58:19.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.819+0000 7f92f5e00640 1 -- 172.21.15.105:0/2709438004 wait complete. 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.821+0000 7f92f5e00640 1 Processor -- start 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.821+0000 7f92f5e00640 1 -- start start 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4003ef0 0x7f92f00c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f01f1f10 0x7f92f015fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92f00c99a0 0x7f92f00b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92f5e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00c6260 con 0x7f92f00c99a0 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92f5e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f01f2480 con 0x7f92d4003ef0 2024-03-23T07:58:19.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92f5e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f01f2760 con 0x7f92f01f1f10 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92f00c99a0 0x7f92f00b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92f00c99a0 0x7f92f00b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54680/0 (socket says 172.21.15.105:54680) 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 -- 172.21.15.105:0/1572645639 learned_addr learned my addr 172.21.15.105:0/1572645639 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ee000640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f01f1f10 0x7f92f015fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92eea00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4003ef0 0x7f92f00c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 -- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f01f1f10 msgr2=0x7f92f015fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f01f1f10 0x7f92f015fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 -- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4003ef0 msgr2=0x7f92f00c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4003ef0 0x7f92f00c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.822+0000 7f92ef400640 1 -- 172.21.15.105:0/1572645639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f01f2a40 con 0x7f92f00c99a0 2024-03-23T07:58:19.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.823+0000 7f92ef400640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92f00c99a0 0x7f92f00b7000 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f92f00cceb0 tx=0x7f92e8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.823+0000 7f92e7400640 1 -- 172.21.15.105:0/1572645639 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8004500 con 0x7f92f00c99a0 2024-03-23T07:58:19.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.823+0000 7f92e7400640 1 -- 172.21.15.105:0/1572645639 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e8003610 con 0x7f92f00c99a0 2024-03-23T07:58:19.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.823+0000 7f92e7400640 1 -- 172.21.15.105:0/1572645639 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80039a0 con 0x7f92f00c99a0 2024-03-23T07:58:19.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.823+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f01f2d20 con 0x7f92f00c99a0 2024-03-23T07:58:19.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.823+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f015ff90 con 0x7f92f00c99a0 2024-03-23T07:58:19.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.824+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f0073390 con 0x7f92f00c99a0 2024-03-23T07:58:19.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.825+0000 7f92e7400640 1 -- 172.21.15.105:0/1572645639 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f92e8003b00 con 0x7f92f00c99a0 2024-03-23T07:58:19.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.826+0000 7f92e7400640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f92cc06ac30 0x7f92cc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:19.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.826+0000 7f92e7400640 1 -- 172.21.15.105:0/1572645639 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f92e80899a0 con 0x7f92f00c99a0 2024-03-23T07:58:19.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.826+0000 7f92eea00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f92cc06ac30 0x7f92cc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:19.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.827+0000 7f92eea00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f92cc06ac30 0x7f92cc06d0f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92e0012350 tx=0x7f92e000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:19.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:19.830+0000 7f92e7400640 1 -- 172.21.15.105:0/1572645639 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f92e8054a10 con 0x7f92f00c99a0 2024-03-23T07:58:20.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.024+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92f00cf900 con 0x7f92f00c99a0 2024-03-23T07:58:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.028+0000 7f92e7400640 1 -- 172.21.15.105:0/1572645639 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f92e8003db0 con 0x7f92f00c99a0 2024-03-23T07:58:20.029 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:20.029 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":44,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:58:14.253586+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:57:51.204124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-23T07:57:52.349446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1e","mappings":[{"from":3,"to":5}]},{"pgid":"6.24","mappings":[{"from":4,"to":7}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]},{"pgid":"6.39","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/3723879365":"2024-03-24T07:58:14.253538+0000","172.21.15.105:6836/3723879365":"2024-03-24T07:58:14.253538+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:58:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.030+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f92cc06ac30 msgr2=0x7f92cc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.030+0000 7f92f5e00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f92cc06ac30 0x7f92cc06d0f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92e0012350 tx=0x7f92e000b040 comp rx=0 tx=0).stop 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.030+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92f00c99a0 msgr2=0x7f92f00b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.030+0000 7f92f5e00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92f00c99a0 0x7f92f00b7000 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f92f00cceb0 tx=0x7f92e8011920 comp rx=0 tx=0).stop 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 shutdown_connections 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92f00c99a0 0x7f92f00b7000 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f92cc06ac30 0x7f92cc06d0f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92f01f1f10 0x7f92f015fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 --2- 172.21.15.105:0/1572645639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92d4003ef0 0x7f92f00c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 >> 172.21.15.105:0/1572645639 conn(0x7f92f00cf350 msgr2=0x7f92f00b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 shutdown_connections 2024-03-23T07:58:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.031+0000 7f92f5e00640 1 -- 172.21.15.105:0/1572645639 wait complete. 2024-03-23T07:58:20.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:57:51.204124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:57:52.349446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T07:58:20.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:58:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.257+0000 7f0893000640 1 Processor -- start 2024-03-23T07:58:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.257+0000 7f0893000640 1 -- start start 2024-03-23T07:58:20.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0893000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f088c06ce90 0x7f088c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0893000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0cdba0 con 0x7f088c0ce930 2024-03-23T07:58:20.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0893000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0cde80 con 0x7f088c0add30 2024-03-23T07:58:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0893000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0ae1c0 con 0x7f088c06ce90 2024-03-23T07:58:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0891000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f088c06ce90 0x7f088c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0891000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f088c06ce90 0x7f088c06d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54692/0 (socket says 172.21.15.105:54692) 2024-03-23T07:58:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0891000640 1 -- 172.21.15.105:0/2922836789 learned_addr learned my addr 172.21.15.105:0/2922836789 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0891000640 1 -- 172.21.15.105:0/2922836789 >> v1:172.21.15.132:6789/0 conn(0x7f088c0add30 legacy=0x7f088c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.259+0000 7f0891000640 1 -- 172.21.15.105:0/2922836789 >> v1:172.21.15.105:6789/0 conn(0x7f088c0ce930 legacy=0x7f088c0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.260+0000 7f0891000640 1 -- 172.21.15.105:0/2922836789 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0ce160 con 0x7f088c06ce90 2024-03-23T07:58:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.261+0000 7f0891000640 1 --2- 172.21.15.105:0/2922836789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c06ce90 0x7f088c06d300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0884009e50 tx=0x7f088402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6277e9a1d36d2ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.261+0000 7f088aa00640 1 -- 172.21.15.105:0/2922836789 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088402cb10 con 0x7f088c06ce90 2024-03-23T07:58:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.261+0000 7f088aa00640 1 -- 172.21.15.105:0/2922836789 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088402cc70 con 0x7f088c06ce90 2024-03-23T07:58:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.261+0000 7f088aa00640 1 -- 172.21.15.105:0/2922836789 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840343f0 con 0x7f088c06ce90 2024-03-23T07:58:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.261+0000 7f0893000640 1 -- 172.21.15.105:0/2922836789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c06ce90 msgr2=0x7f088c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.261+0000 7f0893000640 1 --2- 172.21.15.105:0/2922836789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c06ce90 0x7f088c06d300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0884009e50 tx=0x7f088402bf70 comp rx=0 tx=0).stop 2024-03-23T07:58:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.262+0000 7f0893000640 1 -- 172.21.15.105:0/2922836789 shutdown_connections 2024-03-23T07:58:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.262+0000 7f0893000640 1 --2- 172.21.15.105:0/2922836789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c06ce90 0x7f088c06d300 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.262+0000 7f0893000640 1 -- 172.21.15.105:0/2922836789 >> 172.21.15.105:0/2922836789 conn(0x7f088c06a0b0 msgr2=0x7f088c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.262+0000 7f0893000640 1 -- 172.21.15.105:0/2922836789 shutdown_connections 2024-03-23T07:58:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.262+0000 7f0893000640 1 -- 172.21.15.105:0/2922836789 wait complete. 2024-03-23T07:58:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.263+0000 7f0893000640 1 Processor -- start 2024-03-23T07:58:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.264+0000 7f0893000640 1 -- start start 2024-03-23T07:58:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.264+0000 7f0893000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f088c06ce90 0x7f088c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.264+0000 7f0893000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c0add30 0x7f088c0a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.264+0000 7f0893000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c0ce930 0x7f088c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.264+0000 7f0893000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0a98a0 con 0x7f088c0add30 2024-03-23T07:58:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.264+0000 7f0893000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1f3980 con 0x7f088c0ce930 2024-03-23T07:58:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.264+0000 7f0893000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1f3c60 con 0x7f088c06ce90 2024-03-23T07:58:20.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c0add30 0x7f088c0a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c0add30 0x7f088c0a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54704/0 (socket says 172.21.15.105:54704) 2024-03-23T07:58:20.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 -- 172.21.15.105:0/878375327 learned_addr learned my addr 172.21.15.105:0/878375327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 -- 172.21.15.105:0/878375327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f088c06ce90 msgr2=0x7f088c153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f088c06ce90 0x7f088c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 -- 172.21.15.105:0/878375327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c0ce930 msgr2=0x7f088c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c0ce930 0x7f088c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 -- 172.21.15.105:0/878375327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884009b30 con 0x7f088c0add30 2024-03-23T07:58:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.265+0000 7f088b400640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c0add30 0x7f088c0a8f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f087800c430 tx=0x7f087800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.266+0000 7f0883e00640 1 -- 172.21.15.105:0/878375327 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878011790 con 0x7f088c0add30 2024-03-23T07:58:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.266+0000 7f0883e00640 1 -- 172.21.15.105:0/878375327 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08780118f0 con 0x7f088c0add30 2024-03-23T07:58:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.266+0000 7f0883e00640 1 -- 172.21.15.105:0/878375327 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878011ce0 con 0x7f088c0add30 2024-03-23T07:58:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.266+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c1f3f40 con 0x7f088c0add30 2024-03-23T07:58:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.266+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c0a9470 con 0x7f088c0add30 2024-03-23T07:58:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.267+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c1f37e0 con 0x7f088c0add30 2024-03-23T07:58:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.268+0000 7f0883e00640 1 -- 172.21.15.105:0/878375327 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f087801c460 con 0x7f088c0add30 2024-03-23T07:58:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.268+0000 7f0883e00640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f086c06ac00 0x7f086c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.269+0000 7f0883e00640 1 -- 172.21.15.105:0/878375327 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f08780895a0 con 0x7f088c0add30 2024-03-23T07:58:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.269+0000 7f088be00640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f086c06ac00 0x7f086c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.269+0000 7f088be00640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f086c06ac00 0x7f086c06d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f087c000f60 tx=0x7f087c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.273+0000 7f0883e00640 1 -- 172.21.15.105:0/878375327 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f088c1f37e0 con 0x7f088c0add30 2024-03-23T07:58:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.468+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f088c1f37e0 con 0x7f088c0add30 2024-03-23T07:58:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.469+0000 7f0883e00640 1 -- 172.21.15.105:0/878375327 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f088c1f37e0 con 0x7f088c0add30 2024-03-23T07:58:20.469 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f086c06ac00 msgr2=0x7f086c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f086c06ac00 0x7f086c06d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f087c000f60 tx=0x7f087c0079b0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c0add30 msgr2=0x7f088c0a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c0add30 0x7f088c0a8f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f087800c430 tx=0x7f087800c8f0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 shutdown_connections 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c0ce930 0x7f088c0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f088c0add30 0x7f088c0a8f30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f086c06ac00 0x7f086c06d0c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 --2- 172.21.15.105:0/878375327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f088c06ce90 0x7f088c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 >> 172.21.15.105:0/878375327 conn(0x7f088c06a0b0 msgr2=0x7f088c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 shutdown_connections 2024-03-23T07:58:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.471+0000 7f0893000640 1 -- 172.21.15.105:0/878375327 wait complete. 2024-03-23T07:58:20.489 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:58:20.528 INFO:tasks.ceph.mds.c.smithi105.stderr:2024-03-23T07:58:20.528+0000 7fdf35000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T07:58:20.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.696+0000 7f20de600640 1 Processor -- start 2024-03-23T07:58:20.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.697+0000 7f20de600640 1 -- start start 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.698+0000 7f20de600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f20d80cd740 0x7f20d80b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.698+0000 7f20de600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d80d0d40 0x7f20d80cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.698+0000 7f20de600640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20d806b0a0 0x7f20d806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.698+0000 7f20de600640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80d20c0 con 0x7f20d806b0a0 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.698+0000 7f20de600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80ce060 con 0x7f20d80cd740 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.698+0000 7f20de600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d80d14b0 con 0x7f20d80d0d40 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20d806b0a0 0x7f20d806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20d806b0a0 0x7f20d806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54710/0 (socket says 172.21.15.105:54710) 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 -- 172.21.15.105:0/3536465620 learned_addr learned my addr 172.21.15.105:0/3536465620 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7400640 1 --2- 172.21.15.105:0/3536465620 >> v2:172.21.15.132:3300/0 conn(0x7f20d80cd740 0x7f20d80b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 -- 172.21.15.105:0/3536465620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d80d0d40 msgr2=0x7f20d80cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20cea00640 1 --2- 172.21.15.105:0/3536465620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d80d0d40 0x7f20d80cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 --2- 172.21.15.105:0/3536465620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d80d0d40 0x7f20d80cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 -- 172.21.15.105:0/3536465620 >> v2:172.21.15.132:3300/0 conn(0x7f20d80cd740 msgr2=0x7f20d80b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:20.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 --2- 172.21.15.105:0/3536465620 >> v2:172.21.15.132:3300/0 conn(0x7f20d80cd740 0x7f20d80b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20d7e00640 1 -- 172.21.15.105:0/3536465620 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80d1790 con 0x7f20d806b0a0 2024-03-23T07:58:20.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.699+0000 7f20cea00640 1 --2- 172.21.15.105:0/3536465620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d80d0d40 0x7f20d80cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:20.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.700+0000 7f20d7e00640 1 --2- 172.21.15.105:0/3536465620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d806b0a0 0x7f20d806b530 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f20d808a100 tx=0x7f20d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b046953b14538b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:20.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.700+0000 7f20d6a00640 1 -- 172.21.15.105:0/3536465620 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d002ca80 con 0x7f20d806b0a0 2024-03-23T07:58:20.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.701+0000 7f20d6a00640 1 -- 172.21.15.105:0/3536465620 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d002cbe0 con 0x7f20d806b0a0 2024-03-23T07:58:20.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.701+0000 7f20d6a00640 1 -- 172.21.15.105:0/3536465620 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d00343f0 con 0x7f20d806b0a0 2024-03-23T07:58:20.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.701+0000 7f20de600640 1 -- 172.21.15.105:0/3536465620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d806b0a0 msgr2=0x7f20d806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.701+0000 7f20de600640 1 --2- 172.21.15.105:0/3536465620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d806b0a0 0x7f20d806b530 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f20d808a100 tx=0x7f20d002bee0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.702+0000 7f20de600640 1 -- 172.21.15.105:0/3536465620 shutdown_connections 2024-03-23T07:58:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.702+0000 7f20de600640 1 --2- 172.21.15.105:0/3536465620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d806b0a0 0x7f20d806b530 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.702+0000 7f20de600640 1 --2- 172.21.15.105:0/3536465620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d80d0d40 0x7f20d80cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.702+0000 7f20de600640 1 --2- 172.21.15.105:0/3536465620 >> v2:172.21.15.132:3300/0 conn(0x7f20d80cd740 0x7f20d80b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.702+0000 7f20de600640 1 -- 172.21.15.105:0/3536465620 >> 172.21.15.105:0/3536465620 conn(0x7f20d80cf5d0 msgr2=0x7f20d80cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.702+0000 7f20de600640 1 -- 172.21.15.105:0/3536465620 shutdown_connections 2024-03-23T07:58:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.702+0000 7f20de600640 1 -- 172.21.15.105:0/3536465620 wait complete. 2024-03-23T07:58:20.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.703+0000 7f20de600640 1 Processor -- start 2024-03-23T07:58:20.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.704+0000 7f20de600640 1 -- start start 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.704+0000 7f20de600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d806b0a0 0x7f20d80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.704+0000 7f20de600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20d80cd740 0x7f20d81f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20de600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d80d0d40 0x7f20d80cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20de600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d80b13e0 con 0x7f20d80d0d40 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20de600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81f2d40 con 0x7f20d80cd740 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20de600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81f3020 con 0x7f20d806b0a0 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d80d0d40 0x7f20d80cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d80d0d40 0x7f20d80cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54718/0 (socket says 172.21.15.105:54718) 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 -- 172.21.15.105:0/1927749076 learned_addr learned my addr 172.21.15.105:0/1927749076 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 -- 172.21.15.105:0/1927749076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d806b0a0 msgr2=0x7f20d80b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7400640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d806b0a0 0x7f20d80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d806b0a0 0x7f20d80b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 -- 172.21.15.105:0/1927749076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20d80cd740 msgr2=0x7f20d81f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20d80cd740 0x7f20d81f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.705+0000 7f20d7e00640 1 -- 172.21.15.105:0/1927749076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d0009b30 con 0x7f20d80d0d40 2024-03-23T07:58:20.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.706+0000 7f20d7e00640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d80d0d40 0x7f20d80cb030 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f20d0036040 tx=0x7f20d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:20.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.706+0000 7f20cfe00640 1 -- 172.21.15.105:0/1927749076 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0009510 con 0x7f20d80d0d40 2024-03-23T07:58:20.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.706+0000 7f20cfe00640 1 -- 172.21.15.105:0/1927749076 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d003e450 con 0x7f20d80d0d40 2024-03-23T07:58:20.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.706+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d81f3600 con 0x7f20d80d0d40 2024-03-23T07:58:20.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.706+0000 7f20cfe00640 1 -- 172.21.15.105:0/1927749076 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d003e800 con 0x7f20d80d0d40 2024-03-23T07:58:20.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.706+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d81f3300 con 0x7f20d80d0d40 2024-03-23T07:58:20.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.707+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d81f3b20 con 0x7f20d80d0d40 2024-03-23T07:58:20.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.708+0000 7f20cfe00640 1 -- 172.21.15.105:0/1927749076 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f20d002cd50 con 0x7f20d80d0d40 2024-03-23T07:58:20.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.709+0000 7f20cfe00640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f20b406ac30 0x7f20b406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:20.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.709+0000 7f20cfe00640 1 -- 172.21.15.105:0/1927749076 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f20d00abe50 con 0x7f20d80d0d40 2024-03-23T07:58:20.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.709+0000 7f20d7400640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f20b406ac30 0x7f20b406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:20.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.710+0000 7f20d7400640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f20b406ac30 0x7f20b406d0f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f20c8006fd0 tx=0x7f20c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:20.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.715+0000 7f20cfe00640 1 -- 172.21.15.105:0/1927749076 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f20d81f3b20 con 0x7f20d80d0d40 2024-03-23T07:58:20.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.896+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20d81f3b20 con 0x7f20d80d0d40 2024-03-23T07:58:20.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.897+0000 7f20cfe00640 1 -- 172.21.15.105:0/1927749076 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20d81f3b20 con 0x7f20d80d0d40 2024-03-23T07:58:20.897 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:58:20.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.899+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f20b406ac30 msgr2=0x7f20b406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f20b406ac30 0x7f20b406d0f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f20c8006fd0 tx=0x7f20c8008040 comp rx=0 tx=0).stop 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d80d0d40 msgr2=0x7f20d80cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d80d0d40 0x7f20d80cb030 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f20d0036040 tx=0x7f20d0004a40 comp rx=0 tx=0).stop 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 shutdown_connections 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d80d0d40 0x7f20d80cb030 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20d80cd740 0x7f20d81f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f20b406ac30 0x7f20b406d0f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 --2- 172.21.15.105:0/1927749076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20d806b0a0 0x7f20d80b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 >> 172.21.15.105:0/1927749076 conn(0x7f20d80cf5d0 msgr2=0x7f20d815f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 shutdown_connections 2024-03-23T07:58:20.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:20.900+0000 7f20de600640 1 -- 172.21.15.105:0/1927749076 wait complete. 2024-03-23T07:58:20.919 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:58:21.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.169+0000 7f17c8000640 1 Processor -- start 2024-03-23T07:58:21.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.169+0000 7f17c8000640 1 -- start start 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.170+0000 7f17c8000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f17c0159830 0x7f17c00cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.170+0000 7f17c8000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17c00ca7b0 0x7f17c00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.170+0000 7f17c8000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c00cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.170+0000 7f17c8000640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c006de50 con 0x7f17c00ca7b0 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.170+0000 7f17c8000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c00cf300 con 0x7f17c0159830 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.170+0000 7f17c8000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0159c10 con 0x7f17c00ce930 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17c00ca7b0 0x7f17c00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17c00ca7b0 0x7f17c00cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54728/0 (socket says 172.21.15.105:54728) 2024-03-23T07:58:21.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 -- 172.21.15.105:0/393958915 learned_addr learned my addr 172.21.15.105:0/393958915 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c6000640 1 --2- 172.21.15.105:0/393958915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c00cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 -- 172.21.15.105:0/393958915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 msgr2=0x7f17c00cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 --2- 172.21.15.105:0/393958915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c00cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 -- 172.21.15.105:0/393958915 >> v2:172.21.15.132:3300/0 conn(0x7f17c0159830 msgr2=0x7f17c00cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c5600640 1 --2- 172.21.15.105:0/393958915 >> v2:172.21.15.132:3300/0 conn(0x7f17c0159830 0x7f17c00cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 --2- 172.21.15.105:0/393958915 >> v2:172.21.15.132:3300/0 conn(0x7f17c0159830 0x7f17c00cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c4c00640 1 -- 172.21.15.105:0/393958915 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00cd840 con 0x7f17c00ca7b0 2024-03-23T07:58:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.171+0000 7f17c6000640 1 --2- 172.21.15.105:0/393958915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c00cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:21.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.172+0000 7f17c4c00640 1 --2- 172.21.15.105:0/393958915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c00ca7b0 0x7f17c00cdcd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f17c006e4e0 tx=0x7f17ac02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fdfb0b08e3968d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:21.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17bfe00640 1 -- 172.21.15.105:0/393958915 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac030850 con 0x7f17c00ca7b0 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17bfe00640 1 -- 172.21.15.105:0/393958915 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac0309b0 con 0x7f17c00ca7b0 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17bfe00640 1 -- 172.21.15.105:0/393958915 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac030d40 con 0x7f17c00ca7b0 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 -- 172.21.15.105:0/393958915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c00ca7b0 msgr2=0x7f17c00cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 --2- 172.21.15.105:0/393958915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c00ca7b0 0x7f17c00cdcd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f17c006e4e0 tx=0x7f17ac02b990 comp rx=0 tx=0).stop 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 -- 172.21.15.105:0/393958915 shutdown_connections 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 --2- 172.21.15.105:0/393958915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c00cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 --2- 172.21.15.105:0/393958915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c00ca7b0 0x7f17c00cdcd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 --2- 172.21.15.105:0/393958915 >> v2:172.21.15.132:3300/0 conn(0x7f17c0159830 0x7f17c00cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 -- 172.21.15.105:0/393958915 >> 172.21.15.105:0/393958915 conn(0x7f17c006a0b0 msgr2=0x7f17c006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.173+0000 7f17c8000640 1 -- 172.21.15.105:0/393958915 shutdown_connections 2024-03-23T07:58:21.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.174+0000 7f17c8000640 1 -- 172.21.15.105:0/393958915 wait complete. 2024-03-23T07:58:21.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.175+0000 7f17c8000640 1 Processor -- start 2024-03-23T07:58:21.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.175+0000 7f17c8000640 1 -- start start 2024-03-23T07:58:21.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.176+0000 7f17c8000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17c00ca7b0 0x7f17c0150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.176+0000 7f17c8000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.176+0000 7f17c8000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c0159830 0x7f17c0149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.176+0000 7f17c8000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00ac1e0 con 0x7f17c0159830 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.176+0000 7f17c8000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0150790 con 0x7f17c00ca7b0 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.176+0000 7f17c8000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0150a30 con 0x7f17c00ce930 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c0159830 0x7f17c0149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c0159830 0x7f17c0149290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54736/0 (socket says 172.21.15.105:54736) 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 -- 172.21.15.105:0/690858809 learned_addr learned my addr 172.21.15.105:0/690858809 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c4c00640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 -- 172.21.15.105:0/690858809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 msgr2=0x7f17c015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 -- 172.21.15.105:0/690858809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17c00ca7b0 msgr2=0x7f17c0150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17c00ca7b0 0x7f17c0150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 -- 172.21.15.105:0/690858809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac0094d0 con 0x7f17c0159830 2024-03-23T07:58:21.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.177+0000 7f17c6000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c0159830 0x7f17c0149290 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f17b800cef0 tx=0x7f17b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:21.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.178+0000 7f17bd600640 1 -- 172.21.15.105:0/690858809 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800d840 con 0x7f17c0159830 2024-03-23T07:58:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.178+0000 7f17bd600640 1 -- 172.21.15.105:0/690858809 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b800d9c0 con 0x7f17c0159830 2024-03-23T07:58:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.178+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c00ab8b0 con 0x7f17c0159830 2024-03-23T07:58:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.178+0000 7f17bd600640 1 -- 172.21.15.105:0/690858809 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800dd70 con 0x7f17c0159830 2024-03-23T07:58:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.178+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c00abe70 con 0x7f17c0159830 2024-03-23T07:58:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.179+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c0073390 con 0x7f17c0159830 2024-03-23T07:58:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.180+0000 7f17bd600640 1 -- 172.21.15.105:0/690858809 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f17b801e3b0 con 0x7f17c0159830 2024-03-23T07:58:21.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.181+0000 7f17bd600640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f17a006ac30 0x7f17a006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.181+0000 7f17bd600640 1 -- 172.21.15.105:0/690858809 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f17b8089f60 con 0x7f17c0159830 2024-03-23T07:58:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.181+0000 7f17c5600640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f17a006ac30 0x7f17a006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.182+0000 7f17c5600640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f17a006ac30 0x7f17a006d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f17c015cd40 tx=0x7f17b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:21.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.186+0000 7f17bd600640 1 -- 172.21.15.105:0/690858809 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f17b8055080 con 0x7f17c0159830 2024-03-23T07:58:21.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.381+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17c00ac320 con 0x7f17c0159830 2024-03-23T07:58:21.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.382+0000 7f17bd600640 1 -- 172.21.15.105:0/690858809 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f17b801a020 con 0x7f17c0159830 2024-03-23T07:58:21.383 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f17a006ac30 msgr2=0x7f17a006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f17a006ac30 0x7f17a006d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f17c015cd40 tx=0x7f17b4008040 comp rx=0 tx=0).stop 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c0159830 msgr2=0x7f17c0149290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c0159830 0x7f17c0149290 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f17b800cef0 tx=0x7f17b800a710 comp rx=0 tx=0).stop 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 shutdown_connections 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c0159830 0x7f17c0149290 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f17a006ac30 0x7f17a006d0f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17c00ce930 0x7f17c015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 --2- 172.21.15.105:0/690858809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17c00ca7b0 0x7f17c0150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 >> 172.21.15.105:0/690858809 conn(0x7f17c006a0b0 msgr2=0x7f17c0156580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 shutdown_connections 2024-03-23T07:58:21.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.385+0000 7f17c8000640 1 -- 172.21.15.105:0/690858809 wait complete. 2024-03-23T07:58:21.404 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T07:58:21.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.608+0000 7f46cc000640 1 Processor -- start 2024-03-23T07:58:21.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.608+0000 7f46cc000640 1 -- start start 2024-03-23T07:58:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.610+0000 7f46cc000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 0x7f46c40ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.610+0000 7f46cc000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f46c40cb660 0x7f46c40cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.610+0000 7f46cc000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c40cf250 con 0x7f46c4070170 2024-03-23T07:58:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.610+0000 7f46cc000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c415bae0 con 0x7f46c40cb660 2024-03-23T07:58:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.610+0000 7f46cc000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c415ae60 con 0x7f46c40ce930 2024-03-23T07:58:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46ca000640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f46c4070170 0x7f46c4070600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49388/0 (socket says 172.21.15.105:49388) 2024-03-23T07:58:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46ca000640 1 -- 172.21.15.105:0/779638905 learned_addr learned my addr 172.21.15.105:0/779638905 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c9600640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 0x7f46c40ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779813808 0 0) 0x7f46c40cf250 con 0x7f46c4070170 2024-03-23T07:58:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ac003520 con 0x7f46c4070170 2024-03-23T07:58:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c8c00640 1 --2- 172.21.15.105:0/779638905 >> v2:172.21.15.132:3300/0 conn(0x7f46c40cb660 0x7f46c40cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2227363360 0 0) 0x7f46ac003520 con 0x7f46c4070170 2024-03-23T07:58:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 msgr2=0x7f46c40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 0x7f46c40ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 >> v2:172.21.15.132:3300/0 conn(0x7f46c40cb660 msgr2=0x7f46c40cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 --2- 172.21.15.105:0/779638905 >> v2:172.21.15.132:3300/0 conn(0x7f46c40cb660 0x7f46c40cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c415b140 con 0x7f46c4070170 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.611+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f46bc0019f0 con 0x7f46c4070170 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c9600640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 0x7f46c40ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 >> v1:172.21.15.105:6789/0 conn(0x7f46c4070170 legacy=0x7f46c4070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c3e00640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46ac003970 0x7f46ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c3e00640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0047c0 0x7f46ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c3e00640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 0x7f46ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c415bae0 con 0x7f46ac003970 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c415ae60 con 0x7f46ac0047c0 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac00ba00 con 0x7f46c40ce930 2024-03-23T07:58:21.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c9600640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46ac003970 0x7f46ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c9600640 1 -- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 msgr2=0x7f46ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c9600640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c40ce930 0x7f46ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c9600640 1 -- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0047c0 msgr2=0x7f46ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c9600640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0047c0 0x7f46ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.612+0000 7f46c9600640 1 -- 172.21.15.105:0/779638905 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac003520 con 0x7f46ac003970 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.613+0000 7f46c9600640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46ac003970 0x7f46ac003d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098a0 tx=0x7f46ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59af0800fdb235e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.614+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8004230 con 0x7f46ac003970 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.614+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8004390 con 0x7f46ac003970 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.614+0000 7f46c3e00640 1 -- 172.21.15.105:0/779638905 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8003830 con 0x7f46ac003970 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.614+0000 7f46cc000640 1 -- 172.21.15.105:0/779638905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46ac003970 msgr2=0x7f46ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.614+0000 7f46cc000640 1 --2- 172.21.15.105:0/779638905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46ac003970 0x7f46ac003d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098a0 tx=0x7f46ac0057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.614+0000 7f46ca000640 1 -- 172.21.15.105:0/779638905 reap_dead start 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.615+0000 7f46cc000640 1 -- 172.21.15.105:0/779638905 shutdown_connections 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.615+0000 7f46cc000640 1 -- 172.21.15.105:0/779638905 >> 172.21.15.105:0/779638905 conn(0x7f46c406a0b0 msgr2=0x7f46c406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.615+0000 7f46cc000640 1 -- 172.21.15.105:0/779638905 shutdown_connections 2024-03-23T07:58:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.615+0000 7f46cc000640 1 -- 172.21.15.105:0/779638905 wait complete. 2024-03-23T07:58:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.617+0000 7f46cc000640 1 Processor -- start 2024-03-23T07:58:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.617+0000 7f46cc000640 1 -- start start 2024-03-23T07:58:21.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46cc000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac003970 0x7f46c41ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46cc000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c406c2b0 0x7f46c406c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46cc000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46c4150b90 0x7f46c4151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46cc000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c406cc70 con 0x7f46c4150b90 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46cc000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41518a0 con 0x7f46ac003970 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46cc000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4151b80 con 0x7f46c406c2b0 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46c4150b90 0x7f46c4151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46c4150b90 0x7f46c4151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54744/0 (socket says 172.21.15.105:54744) 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 -- 172.21.15.105:0/2690158311 learned_addr learned my addr 172.21.15.105:0/2690158311 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 -- 172.21.15.105:0/2690158311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c406c2b0 msgr2=0x7f46c406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c406c2b0 0x7f46c406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 -- 172.21.15.105:0/2690158311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac003970 msgr2=0x7f46c41ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac003970 0x7f46c41ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.618+0000 7f46ca000640 1 -- 172.21.15.105:0/2690158311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c4155280 con 0x7f46c4150b90 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.619+0000 7f46ca000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46c4150b90 0x7f46c4151010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f46c40cf540 tx=0x7f46bc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.619+0000 7f46c1600640 1 -- 172.21.15.105:0/2690158311 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00a610 con 0x7f46c4150b90 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.619+0000 7f46c1600640 1 -- 172.21.15.105:0/2690158311 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc00a790 con 0x7f46c4150b90 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.619+0000 7f46c1600640 1 -- 172.21.15.105:0/2690158311 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00ab40 con 0x7f46c4150b90 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.619+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c4151e60 con 0x7f46c4150b90 2024-03-23T07:58:21.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.619+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c4152420 con 0x7f46c4150b90 2024-03-23T07:58:21.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.620+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c40cae10 con 0x7f46c4150b90 2024-03-23T07:58:21.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.621+0000 7f46c1600640 1 -- 172.21.15.105:0/2690158311 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f46bc00aca0 con 0x7f46c4150b90 2024-03-23T07:58:21.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.622+0000 7f46c1600640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f46a406ac30 0x7f46a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:21.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.622+0000 7f46c1600640 1 -- 172.21.15.105:0/2690158311 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f46bc088a90 con 0x7f46c4150b90 2024-03-23T07:58:21.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.622+0000 7f46c9600640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f46a406ac30 0x7f46a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:21.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.623+0000 7f46c9600640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f46a406ac30 0x7f46a406d0f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098a0 tx=0x7f46b8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:21.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.626+0000 7f46c1600640 1 -- 172.21.15.105:0/2690158311 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f46bc053af0 con 0x7f46c4150b90 2024-03-23T07:58:21.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.817+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46c4073390 con 0x7f46c4150b90 2024-03-23T07:58:21.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.818+0000 7f46c1600640 1 -- 172.21.15.105:0/2690158311 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v44) v1 ==== 80+0+124 (secure 0 0 0) 0x7f46bc019080 con 0x7f46c4150b90 2024-03-23T07:58:21.819 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6837/3723879365 2024-03-24T07:58:14.253538+0000 2024-03-23T07:58:21.819 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6836/3723879365 2024-03-24T07:58:14.253538+0000 2024-03-23T07:58:21.819 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2024-03-23T07:58:21.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f46a406ac30 msgr2=0x7f46a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f46a406ac30 0x7f46a406d0f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46b80098a0 tx=0x7f46b8030000 comp rx=0 tx=0).stop 2024-03-23T07:58:21.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46c4150b90 msgr2=0x7f46c4151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:21.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46c4150b90 0x7f46c4151010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f46c40cf540 tx=0x7f46bc00ccd0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 shutdown_connections 2024-03-23T07:58:21.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46c4150b90 0x7f46c4151010 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f46a406ac30 0x7f46a406d0f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46c406c2b0 0x7f46c406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 --2- 172.21.15.105:0/2690158311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac003970 0x7f46c41ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:21.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 >> 172.21.15.105:0/2690158311 conn(0x7f46c406a0b0 msgr2=0x7f46c415d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:21.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 shutdown_connections 2024-03-23T07:58:21.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:21.821+0000 7f46cc000640 1 -- 172.21.15.105:0/2690158311 wait complete. 2024-03-23T07:58:21.838 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-23T07:58:21.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:21.839 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-23T07:58:21.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:58:21.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:21.937 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T07:58:21.984 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:58:21.984 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:22.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T07:58:22.039 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:58:22.039 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:22.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053428,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T07:58:22.095 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:58:22.095 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:22.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T07:58:22.152 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.361+0000 7fa894400640 1 Processor -- start 2024-03-23T07:58:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.361+0000 7fa894400640 1 -- start start 2024-03-23T07:58:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.362+0000 7fa894400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.362+0000 7fa894400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa88c0b1db0 0x7fa88c06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.362+0000 7fa894400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0b38a0 con 0x7fa88c160800 2024-03-23T07:58:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.362+0000 7fa894400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c0b3b80 con 0x7fa88c0b1db0 2024-03-23T07:58:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.362+0000 7fa894400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0b3e60 con 0x7fa88c13faa0 2024-03-23T07:58:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.363+0000 7fa891a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.363+0000 7fa891a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:42580/0 (socket says 172.21.15.105:42580) 2024-03-23T07:58:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.363+0000 7fa891a00640 1 -- 172.21.15.105:0/199870348 learned_addr learned my addr 172.21.15.105:0/199870348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.364+0000 7fa892400640 1 --2- 172.21.15.105:0/199870348 >> v2:172.21.15.132:3300/0 conn(0x7fa88c0b1db0 0x7fa88c06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.364+0000 7fa891a00640 1 -- 172.21.15.105:0/199870348 >> v2:172.21.15.132:3300/0 conn(0x7fa88c0b1db0 msgr2=0x7fa88c06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.364+0000 7fa891a00640 1 --2- 172.21.15.105:0/199870348 >> v2:172.21.15.132:3300/0 conn(0x7fa88c0b1db0 0x7fa88c06c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.364+0000 7fa891a00640 1 -- 172.21.15.105:0/199870348 >> v1:172.21.15.132:6789/0 conn(0x7fa88c160800 legacy=0x7fa88c0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.364+0000 7fa891a00640 1 -- 172.21.15.105:0/199870348 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0b4140 con 0x7fa88c13faa0 2024-03-23T07:58:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.366+0000 7fa891a00640 1 --2- 172.21.15.105:0/199870348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c160f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa880009870 tx=0x7fa88002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=343d4bd27787aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.367+0000 7fa88be00640 1 -- 172.21.15.105:0/199870348 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880030840 con 0x7fa88c13faa0 2024-03-23T07:58:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.367+0000 7fa88be00640 1 -- 172.21.15.105:0/199870348 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8800309a0 con 0x7fa88c13faa0 2024-03-23T07:58:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.367+0000 7fa88be00640 1 -- 172.21.15.105:0/199870348 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880030d30 con 0x7fa88c13faa0 2024-03-23T07:58:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.367+0000 7fa894400640 1 -- 172.21.15.105:0/199870348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 msgr2=0x7fa88c160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.367+0000 7fa894400640 1 --2- 172.21.15.105:0/199870348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c160f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa880009870 tx=0x7fa88002b980 comp rx=0 tx=0).stop 2024-03-23T07:58:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.368+0000 7fa894400640 1 -- 172.21.15.105:0/199870348 shutdown_connections 2024-03-23T07:58:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.368+0000 7fa894400640 1 --2- 172.21.15.105:0/199870348 >> v2:172.21.15.132:3300/0 conn(0x7fa88c0b1db0 0x7fa88c06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.368+0000 7fa894400640 1 --2- 172.21.15.105:0/199870348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c160f30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.368+0000 7fa894400640 1 -- 172.21.15.105:0/199870348 >> 172.21.15.105:0/199870348 conn(0x7fa88c06a3d0 msgr2=0x7fa88c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.368+0000 7fa894400640 1 -- 172.21.15.105:0/199870348 shutdown_connections 2024-03-23T07:58:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.368+0000 7fa894400640 1 -- 172.21.15.105:0/199870348 wait complete. 2024-03-23T07:58:22.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.369+0000 7fa894400640 1 Processor -- start 2024-03-23T07:58:22.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.370+0000 7fa894400640 1 -- start start 2024-03-23T07:58:22.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.370+0000 7fa894400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b1db0 0x7fa88c141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.370+0000 7fa894400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c1f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.370+0000 7fa894400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c160800 0x7fa88c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.370+0000 7fa894400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c140a10 con 0x7fa88c0b1db0 2024-03-23T07:58:22.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.370+0000 7fa894400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c140cb0 con 0x7fa88c160800 2024-03-23T07:58:22.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.370+0000 7fa894400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c140f90 con 0x7fa88c13faa0 2024-03-23T07:58:22.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b1db0 0x7fa88c141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b1db0 0x7fa88c141d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54756/0 (socket says 172.21.15.105:54756) 2024-03-23T07:58:22.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 -- 172.21.15.105:0/3072957728 learned_addr learned my addr 172.21.15.105:0/3072957728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:22.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 -- 172.21.15.105:0/3072957728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 msgr2=0x7fa88c1f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c1f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 -- 172.21.15.105:0/3072957728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c160800 msgr2=0x7fa88c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c160800 0x7fa88c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.371+0000 7fa891a00640 1 -- 172.21.15.105:0/3072957728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800094d0 con 0x7fa88c0b1db0 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.372+0000 7fa891a00640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b1db0 0x7fa88c141d50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa880000c00 tx=0x7fa880004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.372+0000 7fa889600640 1 -- 172.21.15.105:0/3072957728 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880039c10 con 0x7fa88c0b1db0 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.372+0000 7fa889600640 1 -- 172.21.15.105:0/3072957728 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880039d70 con 0x7fa88c0b1db0 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.372+0000 7fa889600640 1 -- 172.21.15.105:0/3072957728 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880043640 con 0x7fa88c0b1db0 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.372+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c141270 con 0x7fa88c0b1db0 2024-03-23T07:58:22.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.372+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c141830 con 0x7fa88c0b1db0 2024-03-23T07:58:22.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.373+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c06d9f0 con 0x7fa88c0b1db0 2024-03-23T07:58:22.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.374+0000 7fa889600640 1 -- 172.21.15.105:0/3072957728 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa8800437a0 con 0x7fa88c0b1db0 2024-03-23T07:58:22.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.375+0000 7fa889600640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa87006ac00 0x7fa87006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.375+0000 7fa889600640 1 -- 172.21.15.105:0/3072957728 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fa88003e070 con 0x7fa88c0b1db0 2024-03-23T07:58:22.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.376+0000 7fa891000640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa87006ac00 0x7fa87006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.376+0000 7fa891000640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa87006ac00 0x7fa87006d0c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000f60 tx=0x7fa87c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:22.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.380+0000 7fa889600640 1 -- 172.21.15.105:0/3072957728 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa8800774a0 con 0x7fa88c0b1db0 2024-03-23T07:58:22.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.610+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa88c141a20 con 0x7fa88c0b1db0 2024-03-23T07:58:22.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.612+0000 7fa889600640 1 -- 172.21.15.105:0/3072957728 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fa88007aac0 con 0x7fa88c0b1db0 2024-03-23T07:58:22.612 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:22.613 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/12048571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":12048571},{"type":"v1","addr":"172.21.15.105:6837","nonce":12048571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:58:18.306711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5253":{"gid":5253,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:22.613 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 22 2024-03-23T07:58:22.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa87006ac00 msgr2=0x7fa87006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa87006ac00 0x7fa87006d0c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000f60 tx=0x7fa87c0079b0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b1db0 msgr2=0x7fa88c141d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b1db0 0x7fa88c141d50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa880000c00 tx=0x7fa880004730 comp rx=0 tx=0).stop 2024-03-23T07:58:22.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 shutdown_connections 2024-03-23T07:58:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c160800 0x7fa88c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa87006ac00 0x7fa87006d0c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa88c13faa0 0x7fa88c1f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 --2- 172.21.15.105:0/3072957728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b1db0 0x7fa88c141d50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.614+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 >> 172.21.15.105:0/3072957728 conn(0x7fa88c06a3d0 msgr2=0x7fa88c13f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.615+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 shutdown_connections 2024-03-23T07:58:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.615+0000 7fa894400640 1 -- 172.21.15.105:0/3072957728 wait complete. 2024-03-23T07:58:22.633 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:58:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.840+0000 7f5790200640 1 Processor -- start 2024-03-23T07:58:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.840+0000 7f5790200640 1 -- start start 2024-03-23T07:58:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f5790200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f578813faa0 0x7f5788160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f5790200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57880b1db0 0x7f578806c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f5790200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57880b38a0 con 0x7f5788160800 2024-03-23T07:58:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f5790200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57880b3b80 con 0x7f578813faa0 2024-03-23T07:58:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f5790200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57880b3e60 con 0x7f57880b1db0 2024-03-23T07:58:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f578d800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f578813faa0 0x7f5788160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f578d800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f578813faa0 0x7f5788160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54768/0 (socket says 172.21.15.105:54768) 2024-03-23T07:58:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f578d800640 1 -- 172.21.15.105:0/1654905662 learned_addr learned my addr 172.21.15.105:0/1654905662 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.842+0000 7f578e200640 1 --2- 172.21.15.105:0/1654905662 >> v2:172.21.15.132:3300/0 conn(0x7f57880b1db0 0x7f578806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.843+0000 7f578d800640 1 -- 172.21.15.105:0/1654905662 >> v2:172.21.15.132:3300/0 conn(0x7f57880b1db0 msgr2=0x7f578806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.843+0000 7f578d800640 1 --2- 172.21.15.105:0/1654905662 >> v2:172.21.15.132:3300/0 conn(0x7f57880b1db0 0x7f578806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.843+0000 7f578d800640 1 -- 172.21.15.105:0/1654905662 >> v1:172.21.15.132:6789/0 conn(0x7f5788160800 legacy=0x7f57880b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.843+0000 7f578d800640 1 -- 172.21.15.105:0/1654905662 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57880b4140 con 0x7f578813faa0 2024-03-23T07:58:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.844+0000 7f578d800640 1 --2- 172.21.15.105:0/1654905662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f5788160f30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57880b2c80 tx=0x7f577c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4c6624305cb15b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.844+0000 7f5787e00640 1 -- 172.21.15.105:0/1654905662 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c031870 con 0x7f578813faa0 2024-03-23T07:58:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.844+0000 7f5787e00640 1 -- 172.21.15.105:0/1654905662 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c0319d0 con 0x7f578813faa0 2024-03-23T07:58:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.844+0000 7f5787e00640 1 -- 172.21.15.105:0/1654905662 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c031d60 con 0x7f578813faa0 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 -- 172.21.15.105:0/1654905662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 msgr2=0x7f5788160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 --2- 172.21.15.105:0/1654905662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f5788160f30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57880b2c80 tx=0x7f577c02b910 comp rx=0 tx=0).stop 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 -- 172.21.15.105:0/1654905662 shutdown_connections 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 --2- 172.21.15.105:0/1654905662 >> v2:172.21.15.132:3300/0 conn(0x7f57880b1db0 0x7f578806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 --2- 172.21.15.105:0/1654905662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f5788160f30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 -- 172.21.15.105:0/1654905662 >> 172.21.15.105:0/1654905662 conn(0x7f578806a3d0 msgr2=0x7f57880b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 -- 172.21.15.105:0/1654905662 shutdown_connections 2024-03-23T07:58:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.845+0000 7f5790200640 1 -- 172.21.15.105:0/1654905662 wait complete. 2024-03-23T07:58:22.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.847+0000 7f5790200640 1 Processor -- start 2024-03-23T07:58:22.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.847+0000 7f5790200640 1 -- start start 2024-03-23T07:58:22.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f5790200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57880b1db0 0x7f5788152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f5790200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f578813df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f5790200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5788160800 0x7f578813e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f5790200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5788154b20 con 0x7f578813faa0 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f5790200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5788142d50 con 0x7f5788160800 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f5790200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57881541f0 con 0x7f57880b1db0 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f578813df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f578813df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54776/0 (socket says 172.21.15.105:54776) 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 -- 172.21.15.105:0/3996590470 learned_addr learned my addr 172.21.15.105:0/3996590470 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578d800640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57880b1db0 0x7f5788152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 -- 172.21.15.105:0/3996590470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57880b1db0 msgr2=0x7f5788152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57880b1db0 0x7f5788152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 -- 172.21.15.105:0/3996590470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5788160800 msgr2=0x7f578813e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5788160800 0x7f578813e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.848+0000 7f578ce00640 1 -- 172.21.15.105:0/3996590470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0094d0 con 0x7f578813faa0 2024-03-23T07:58:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.849+0000 7f578ce00640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f578813df90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f577400b3f0 tx=0x7f577400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.849+0000 7f5785600640 1 -- 172.21.15.105:0/3996590470 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774011830 con 0x7f578813faa0 2024-03-23T07:58:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.849+0000 7f5785600640 1 -- 172.21.15.105:0/3996590470 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5774011990 con 0x7f578813faa0 2024-03-23T07:58:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.849+0000 7f5785600640 1 -- 172.21.15.105:0/3996590470 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774011d40 con 0x7f578813faa0 2024-03-23T07:58:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.849+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57881544d0 con 0x7f578813faa0 2024-03-23T07:58:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.849+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5788153230 con 0x7f578813faa0 2024-03-23T07:58:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.850+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5788154cd0 con 0x7f578813faa0 2024-03-23T07:58:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.851+0000 7f5785600640 1 -- 172.21.15.105:0/3996590470 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f577401a400 con 0x7f578813faa0 2024-03-23T07:58:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.852+0000 7f5785600640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f576806ac30 0x7f576806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.852+0000 7f5785600640 1 -- 172.21.15.105:0/3996590470 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f57740893a0 con 0x7f578813faa0 2024-03-23T07:58:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.852+0000 7f578d800640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f576806ac30 0x7f576806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.853+0000 7f578d800640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f576806ac30 0x7f576806d0f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f577c036000 tx=0x7f577c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:22.856+0000 7f5785600640 1 -- 172.21.15.105:0/3996590470 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5788154cd0 con 0x7f578813faa0 2024-03-23T07:58:23.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.050+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5788073390 con 0x7f578813faa0 2024-03-23T07:58:23.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.053+0000 7f5785600640 1 -- 172.21.15.105:0/3996590470 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f5774054360 con 0x7f578813faa0 2024-03-23T07:58:23.054 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:23.055 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":44,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:58:14.253586+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:57:51.204124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-23T07:57:52.349446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1e","mappings":[{"from":3,"to":5}]},{"pgid":"6.24","mappings":[{"from":4,"to":7}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]},{"pgid":"6.39","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/3723879365":"2024-03-24T07:58:14.253538+0000","172.21.15.105:6836/3723879365":"2024-03-24T07:58:14.253538+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f576806ac30 msgr2=0x7f576806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f576806ac30 0x7f576806d0f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f577c036000 tx=0x7f577c03c040 comp rx=0 tx=0).stop 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 msgr2=0x7f578813df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f578813df90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f577400b3f0 tx=0x7f577400b8b0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 shutdown_connections 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5788160800 0x7f578813e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f578813faa0 0x7f578813df90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f576806ac30 0x7f576806d0f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 --2- 172.21.15.105:0/3996590470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57880b1db0 0x7f5788152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 >> 172.21.15.105:0/3996590470 conn(0x7f578806a3d0 msgr2=0x7f57880aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 shutdown_connections 2024-03-23T07:58:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.056+0000 7f5790200640 1 -- 172.21.15.105:0/3996590470 wait complete. 2024-03-23T07:58:23.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T07:56:06.113298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T07:57:51.204124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T07:57:52.349446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T07:58:23.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T07:58:23.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.278+0000 7f3489200640 1 Processor -- start 2024-03-23T07:58:23.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.278+0000 7f3489200640 1 -- start start 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.279+0000 7f3489200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34841456a0 0x7f34840ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3489200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3484145c20 0x7f34840cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3489200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348406d390 con 0x7f3484070cd0 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3489200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34840ad890 con 0x7f3484145c20 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3489200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3484145a80 con 0x7f34841456a0 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3482800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3484145c20 0x7f34840cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3482800640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3484145c20 0x7f34840cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54778/0 (socket says 172.21.15.105:54778) 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3482800640 1 -- 172.21.15.105:0/1038128591 learned_addr learned my addr 172.21.15.105:0/1038128591 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3481e00640 1 --2- 172.21.15.105:0/1038128591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34841456a0 0x7f34840ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3482800640 1 -- 172.21.15.105:0/1038128591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34841456a0 msgr2=0x7f34840ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3482800640 1 --2- 172.21.15.105:0/1038128591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34841456a0 0x7f34840ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.280+0000 7f3482800640 1 -- 172.21.15.105:0/1038128591 >> v1:172.21.15.132:6789/0 conn(0x7f3484070cd0 legacy=0x7f34840ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.281+0000 7f3482800640 1 -- 172.21.15.105:0/1038128591 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348406d670 con 0x7f3484145c20 2024-03-23T07:58:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.281+0000 7f3481e00640 1 --2- 172.21.15.105:0/1038128591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34841456a0 0x7f34840ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.282+0000 7f3482800640 1 --2- 172.21.15.105:0/1038128591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484145c20 0x7f34840cee70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f347c009e80 tx=0x7f347c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fce1a99a965f61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.282+0000 7f3481400640 1 -- 172.21.15.105:0/1038128591 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c02cae0 con 0x7f3484145c20 2024-03-23T07:58:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.282+0000 7f3481400640 1 -- 172.21.15.105:0/1038128591 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c02cc40 con 0x7f3484145c20 2024-03-23T07:58:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.282+0000 7f3481400640 1 -- 172.21.15.105:0/1038128591 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0343f0 con 0x7f3484145c20 2024-03-23T07:58:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 -- 172.21.15.105:0/1038128591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484145c20 msgr2=0x7f34840cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 --2- 172.21.15.105:0/1038128591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484145c20 0x7f34840cee70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f347c009e80 tx=0x7f347c02bf40 comp rx=0 tx=0).stop 2024-03-23T07:58:23.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 -- 172.21.15.105:0/1038128591 shutdown_connections 2024-03-23T07:58:23.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 --2- 172.21.15.105:0/1038128591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484145c20 0x7f34840cee70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 --2- 172.21.15.105:0/1038128591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34841456a0 0x7f34840ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 -- 172.21.15.105:0/1038128591 >> 172.21.15.105:0/1038128591 conn(0x7f348406a0b0 msgr2=0x7f348406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:23.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 -- 172.21.15.105:0/1038128591 shutdown_connections 2024-03-23T07:58:23.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.283+0000 7f3489200640 1 -- 172.21.15.105:0/1038128591 wait complete. 2024-03-23T07:58:23.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.285+0000 7f3489200640 1 Processor -- start 2024-03-23T07:58:23.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.285+0000 7f3489200640 1 -- start start 2024-03-23T07:58:23.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f3489200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3484070cd0 0x7f34841f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f3489200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34841456a0 0x7f34841f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f3489200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3484145c20 0x7f348415da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f3489200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34841f35e0 con 0x7f34841456a0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f3489200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348415df70 con 0x7f3484070cd0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f3489200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34841f25e0 con 0x7f3484145c20 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34841456a0 0x7f34841f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34841456a0 0x7f34841f30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54780/0 (socket says 172.21.15.105:54780) 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 -- 172.21.15.105:0/1560845986 learned_addr learned my addr 172.21.15.105:0/1560845986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 -- 172.21.15.105:0/1560845986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3484145c20 msgr2=0x7f348415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3484145c20 0x7f348415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f3481e00640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3484070cd0 0x7f34841f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 -- 172.21.15.105:0/1560845986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3484070cd0 msgr2=0x7f34841f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3484070cd0 0x7f34841f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.286+0000 7f347be00640 1 -- 172.21.15.105:0/1560845986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c009b30 con 0x7f34841456a0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.287+0000 7f347be00640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34841456a0 0x7f34841f30a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f346c00c400 tx=0x7f346c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.287+0000 7f347a000640 1 -- 172.21.15.105:0/1560845986 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c0052d0 con 0x7f34841456a0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.287+0000 7f347a000640 1 -- 172.21.15.105:0/1560845986 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c005430 con 0x7f34841456a0 2024-03-23T07:58:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.287+0000 7f347a000640 1 -- 172.21.15.105:0/1560845986 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c003830 con 0x7f34841456a0 2024-03-23T07:58:23.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.287+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348415e220 con 0x7f34841456a0 2024-03-23T07:58:23.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.287+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34841f28a0 con 0x7f34841456a0 2024-03-23T07:58:23.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.288+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3484142a90 con 0x7f34841456a0 2024-03-23T07:58:23.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.289+0000 7f347a000640 1 -- 172.21.15.105:0/1560845986 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f346c003990 con 0x7f34841456a0 2024-03-23T07:58:23.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.290+0000 7f347a000640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f346006ac00 0x7f346006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.290+0000 7f347a000640 1 -- 172.21.15.105:0/1560845986 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f346c088b80 con 0x7f34841456a0 2024-03-23T07:58:23.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.290+0000 7f3481e00640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f346006ac00 0x7f346006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.291+0000 7f3481e00640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f346006ac00 0x7f346006d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3474008850 tx=0x7f3474008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:23.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.295+0000 7f347a000640 1 -- 172.21.15.105:0/1560845986 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f346c053b40 con 0x7f34841456a0 2024-03-23T07:58:23.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.491+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34840b1a00 con 0x7f34841456a0 2024-03-23T07:58:23.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.492+0000 7f347a000640 1 -- 172.21.15.105:0/1560845986 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f346c088e60 con 0x7f34841456a0 2024-03-23T07:58:23.493 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f346006ac00 msgr2=0x7f346006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f346006ac00 0x7f346006d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3474008850 tx=0x7f3474008040 comp rx=0 tx=0).stop 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34841456a0 msgr2=0x7f34841f30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34841456a0 0x7f34841f30a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f346c00c400 tx=0x7f346c00c8c0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 shutdown_connections 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f346006ac00 0x7f346006d0c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3484145c20 0x7f348415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34841456a0 0x7f34841f30a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 --2- 172.21.15.105:0/1560845986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3484070cd0 0x7f34841f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 >> 172.21.15.105:0/1560845986 conn(0x7f348406a0b0 msgr2=0x7f3484160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 shutdown_connections 2024-03-23T07:58:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.495+0000 7f3489200640 1 -- 172.21.15.105:0/1560845986 wait complete. 2024-03-23T07:58:23.514 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T07:58:23.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.717+0000 7fa8d0e00640 1 Processor -- start 2024-03-23T07:58:23.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.717+0000 7fa8d0e00640 1 -- start start 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.718+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc0b2030 0x7fa8cc069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.718+0000 7fa8d0e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8cc06a1d0 0x7fa8cc06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.718+0000 7fa8d0e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0b33b0 con 0x7fa8cc0b1c50 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.718+0000 7fa8d0e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b2f90 con 0x7fa8cc06a1d0 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.718+0000 7fa8d0e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc06a660 con 0x7fa8cc0b2030 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.718+0000 7fa8cbe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8cc06a1d0 0x7fa8cc06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.719+0000 7fa8cbe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8cc06a1d0 0x7fa8cc06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54782/0 (socket says 172.21.15.105:54782) 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.719+0000 7fa8cbe00640 1 -- 172.21.15.105:0/3261016384 learned_addr learned my addr 172.21.15.105:0/3261016384 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:23.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.719+0000 7fa8c2a00640 1 --2- 172.21.15.105:0/3261016384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc0b2030 0x7fa8cc069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.719+0000 7fa8cbe00640 1 -- 172.21.15.105:0/3261016384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc0b2030 msgr2=0x7fa8cc069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:23.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.719+0000 7fa8cbe00640 1 --2- 172.21.15.105:0/3261016384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc0b2030 0x7fa8cc069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.719+0000 7fa8cbe00640 1 -- 172.21.15.105:0/3261016384 >> v1:172.21.15.132:6789/0 conn(0x7fa8cc0b1c50 legacy=0x7fa8cc0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:23.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.719+0000 7fa8cbe00640 1 -- 172.21.15.105:0/3261016384 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc159410 con 0x7fa8cc06a1d0 2024-03-23T07:58:23.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.720+0000 7fa8cbe00640 1 --2- 172.21.15.105:0/3261016384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc06a1d0 0x7fa8cc06ffe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009870 tx=0x7fa8c402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61848d0304b9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:23.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.721+0000 7fa8caa00640 1 -- 172.21.15.105:0/3261016384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40317c0 con 0x7fa8cc06a1d0 2024-03-23T07:58:23.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.721+0000 7fa8caa00640 1 -- 172.21.15.105:0/3261016384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c4031920 con 0x7fa8cc06a1d0 2024-03-23T07:58:23.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.721+0000 7fa8caa00640 1 -- 172.21.15.105:0/3261016384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4031cb0 con 0x7fa8cc06a1d0 2024-03-23T07:58:23.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.721+0000 7fa8d0e00640 1 -- 172.21.15.105:0/3261016384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc06a1d0 msgr2=0x7fa8cc06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.721+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/3261016384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc06a1d0 0x7fa8cc06ffe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009870 tx=0x7fa8c402b860 comp rx=0 tx=0).stop 2024-03-23T07:58:23.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.722+0000 7fa8d0e00640 1 -- 172.21.15.105:0/3261016384 shutdown_connections 2024-03-23T07:58:23.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.722+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/3261016384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc06a1d0 0x7fa8cc06ffe0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.722+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/3261016384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc0b2030 0x7fa8cc069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.722+0000 7fa8d0e00640 1 -- 172.21.15.105:0/3261016384 >> 172.21.15.105:0/3261016384 conn(0x7fa8cc0b01b0 msgr2=0x7fa8cc0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:23.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.722+0000 7fa8d0e00640 1 -- 172.21.15.105:0/3261016384 shutdown_connections 2024-03-23T07:58:23.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.722+0000 7fa8d0e00640 1 -- 172.21.15.105:0/3261016384 wait complete. 2024-03-23T07:58:23.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.723+0000 7fa8d0e00640 1 Processor -- start 2024-03-23T07:58:23.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.724+0000 7fa8d0e00640 1 -- start start 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.724+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc06a1d0 0x7fa8cc14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.724+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8cc0b1c50 0x7fa8cc1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc0b2030 0x7fa8cc15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8d0e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15b920 con 0x7fa8cc0b2030 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8d0e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc159c70 con 0x7fa8cc0b1c50 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8d0e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc159f50 con 0x7fa8cc06a1d0 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc0b2030 0x7fa8cc15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc0b2030 0x7fa8cc15b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54796/0 (socket says 172.21.15.105:54796) 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 -- 172.21.15.105:0/2681748013 learned_addr learned my addr 172.21.15.105:0/2681748013 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cb400640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc06a1d0 0x7fa8cc14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8c2a00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8cc0b1c50 0x7fa8cc1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 -- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc06a1d0 msgr2=0x7fa8cc14cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc06a1d0 0x7fa8cc14cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 -- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8cc0b1c50 msgr2=0x7fa8cc1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8cc0b1c50 0x7fa8cc1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cbe00640 1 -- 172.21.15.105:0/2681748013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40094d0 con 0x7fa8cc0b2030 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8cb400640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc06a1d0 0x7fa8cc14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.725+0000 7fa8c2a00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8cc0b1c50 0x7fa8cc1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.726+0000 7fa8cbe00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc0b2030 0x7fa8cc15b3e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40090b0 tx=0x7fa8c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.726+0000 7fa8c3e00640 1 -- 172.21.15.105:0/2681748013 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4032860 con 0x7fa8cc0b2030 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.726+0000 7fa8c3e00640 1 -- 172.21.15.105:0/2681748013 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c40329e0 con 0x7fa8cc0b2030 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.726+0000 7fa8c3e00640 1 -- 172.21.15.105:0/2681748013 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4032d90 con 0x7fa8cc0b2030 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.726+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc15a230 con 0x7fa8cc0b2030 2024-03-23T07:58:23.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.726+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc15a7f0 con 0x7fa8cc0b2030 2024-03-23T07:58:23.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.727+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc06e120 con 0x7fa8cc0b2030 2024-03-23T07:58:23.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.728+0000 7fa8c3e00640 1 -- 172.21.15.105:0/2681748013 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa8c404e450 con 0x7fa8cc0b2030 2024-03-23T07:58:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.729+0000 7fa8c3e00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa8a806ac00 0x7fa8a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.729+0000 7fa8c3e00640 1 -- 172.21.15.105:0/2681748013 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fa8c40ab640 con 0x7fa8cc0b2030 2024-03-23T07:58:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.729+0000 7fa8cb400640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa8a806ac00 0x7fa8a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:23.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.730+0000 7fa8cb400640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa8a806ac00 0x7fa8a806d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc09a2f0 tx=0x7fa8bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:23.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.734+0000 7fa8c3e00640 1 -- 172.21.15.105:0/2681748013 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa8c40766a0 con 0x7fa8cc0b2030 2024-03-23T07:58:23.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.928+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8cc15a9e0 con 0x7fa8cc0b2030 2024-03-23T07:58:23.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.929+0000 7fa8c3e00640 1 -- 172.21.15.105:0/2681748013 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8c4079cc0 con 0x7fa8cc0b2030 2024-03-23T07:58:23.930 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-23T07:58:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa8a806ac00 msgr2=0x7fa8a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa8a806ac00 0x7fa8a806d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc09a2f0 tx=0x7fa8bc009040 comp rx=0 tx=0).stop 2024-03-23T07:58:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc0b2030 msgr2=0x7fa8cc15b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc0b2030 0x7fa8cc15b3e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40090b0 tx=0x7fa8c40046c0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 shutdown_connections 2024-03-23T07:58:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc0b2030 0x7fa8cc15b3e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8cc0b1c50 0x7fa8cc1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa8a806ac00 0x7fa8a806d0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 --2- 172.21.15.105:0/2681748013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8cc06a1d0 0x7fa8cc14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.932+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 >> 172.21.15.105:0/2681748013 conn(0x7fa8cc0b01b0 msgr2=0x7fa8cc0b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.933+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 shutdown_connections 2024-03-23T07:58:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:23.933+0000 7fa8d0e00640 1 -- 172.21.15.105:0/2681748013 wait complete. 2024-03-23T07:58:23.952 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T07:58:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.157+0000 7fb3a4e00640 1 Processor -- start 2024-03-23T07:58:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.157+0000 7fb3a4e00640 1 -- start start 2024-03-23T07:58:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.159+0000 7fb3a4e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 0x7fb3a00cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.159+0000 7fb3a4e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a0159c10 con 0x7fb3a00ca7b0 2024-03-23T07:58:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.159+0000 7fb3a4e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a00cd840 con 0x7fb3a00ce930 2024-03-23T07:58:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.159+0000 7fb3a4e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a006de50 con 0x7fb3a0159830 2024-03-23T07:58:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.159+0000 7fb39d400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb3a00ca7b0 0x7fb3a00cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49398/0 (socket says 172.21.15.105:49398) 2024-03-23T07:58:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.159+0000 7fb39d400640 1 -- 172.21.15.105:0/3951787566 learned_addr learned my addr 172.21.15.105:0/3951787566 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.159+0000 7fb39de00640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 0x7fb3a00cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806170797 0 0) 0x7fb3a0159c10 con 0x7fb3a00ca7b0 2024-03-23T07:58:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb384003520 con 0x7fb3a00ca7b0 2024-03-23T07:58:24.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351654733 0 0) 0x7fb384003520 con 0x7fb3a00ca7b0 2024-03-23T07:58:24.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 msgr2=0x7fb3a00cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:24.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 0x7fb3a00cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 >> v1:172.21.15.132:6789/0 conn(0x7fb3a00ce930 legacy=0x7fb3a00b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00ccc30 con 0x7fb3a00ca7b0 2024-03-23T07:58:24.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fb388001670 con 0x7fb3a00ca7b0 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 >> v1:172.21.15.105:6789/0 conn(0x7fb3a00ca7b0 legacy=0x7fb3a00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 0x7fb3840039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb384003f20 0x7fb3840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb384004d00 0x7fb384005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38400dcd0 con 0x7fb384003f20 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38400de40 con 0x7fb384004d00 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.160+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38400e000 con 0x7fb3a0159830 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39d400640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb384003f20 0x7fb3840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39de00640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 0x7fb3840039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39e800640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb384004d00 0x7fb384005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39d400640 1 -- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 msgr2=0x7fb3840039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39d400640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3a0159830 0x7fb3840039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39d400640 1 -- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb384004d00 msgr2=0x7fb384005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39d400640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb384004d00 0x7fb384005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39d400640 1 -- 172.21.15.105:0/3951787566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384003520 con 0x7fb384003f20 2024-03-23T07:58:24.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.161+0000 7fb39e800640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb384004d00 0x7fb384005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:24.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.162+0000 7fb39d400640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb384003f20 0x7fb3840047c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb38800c240 tx=0x7fb3840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06af9399c782f74 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.162+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388005ad0 con 0x7fb384003f20 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388005c30 con 0x7fb384003f20 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb39ca00640 1 -- 172.21.15.105:0/3951787566 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38802caf0 con 0x7fb384003f20 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb3a4e00640 1 -- 172.21.15.105:0/3951787566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb384003f20 msgr2=0x7fb3840047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb3a4e00640 1 --2- 172.21.15.105:0/3951787566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb384003f20 0x7fb3840047c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb38800c240 tx=0x7fb3840057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb39e800640 1 -- 172.21.15.105:0/3951787566 reap_dead start 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb3a4e00640 1 -- 172.21.15.105:0/3951787566 shutdown_connections 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb3a4e00640 1 -- 172.21.15.105:0/3951787566 >> 172.21.15.105:0/3951787566 conn(0x7fb3a006a0b0 msgr2=0x7fb3a006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb3a4e00640 1 -- 172.21.15.105:0/3951787566 shutdown_connections 2024-03-23T07:58:24.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.163+0000 7fb3a4e00640 1 -- 172.21.15.105:0/3951787566 wait complete. 2024-03-23T07:58:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.165+0000 7fb3a4e00640 1 Processor -- start 2024-03-23T07:58:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.165+0000 7fb3a4e00640 1 -- start start 2024-03-23T07:58:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb3a4e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb384003f20 0x7fb3a01f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb3a4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3a00acdb0 0x7fb3a015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb3a4e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3a00b1390 0x7fb3a00b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb3a4e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a00b1d50 con 0x7fb3a00acdb0 2024-03-23T07:58:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb3a4e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a0151a30 con 0x7fb3a00b1390 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb3a4e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a015c2f0 con 0x7fb384003f20 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb39d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3a00acdb0 0x7fb3a015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb39d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3a00acdb0 0x7fb3a015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54824/0 (socket says 172.21.15.105:54824) 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.166+0000 7fb39d400640 1 -- 172.21.15.105:0/2118476028 learned_addr learned my addr 172.21.15.105:0/2118476028 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39de00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb384003f20 0x7fb3a01f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39e800640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3a00b1390 0x7fb3a00b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39d400640 1 -- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb384003f20 msgr2=0x7fb3a01f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39d400640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb384003f20 0x7fb3a01f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39d400640 1 -- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3a00b1390 msgr2=0x7fb3a00b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39d400640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3a00b1390 0x7fb3a00b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39d400640 1 -- 172.21.15.105:0/2118476028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a015c5d0 con 0x7fb3a00acdb0 2024-03-23T07:58:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.167+0000 7fb39d400640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3a00acdb0 0x7fb3a015b9a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb38802c450 tx=0x7fb388001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.168+0000 7fb396000640 1 -- 172.21.15.105:0/2118476028 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38803c040 con 0x7fb3a00acdb0 2024-03-23T07:58:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.168+0000 7fb396000640 1 -- 172.21.15.105:0/2118476028 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38802ccc0 con 0x7fb3a00acdb0 2024-03-23T07:58:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.168+0000 7fb396000640 1 -- 172.21.15.105:0/2118476028 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880072b0 con 0x7fb3a00acdb0 2024-03-23T07:58:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.168+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a015c8b0 con 0x7fb3a00acdb0 2024-03-23T07:58:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.168+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a015bee0 con 0x7fb3a00acdb0 2024-03-23T07:58:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.169+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a015c0a0 con 0x7fb3a00acdb0 2024-03-23T07:58:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.170+0000 7fb396000640 1 -- 172.21.15.105:0/2118476028 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb388032070 con 0x7fb3a00acdb0 2024-03-23T07:58:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.171+0000 7fb396000640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb37c06ac00 0x7fb37c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.171+0000 7fb396000640 1 -- 172.21.15.105:0/2118476028 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fb3880aabf0 con 0x7fb3a00acdb0 2024-03-23T07:58:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.171+0000 7fb39de00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb37c06ac00 0x7fb37c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.172+0000 7fb39de00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb37c06ac00 0x7fb37c06d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb390007520 tx=0x7fb39000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:24.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.176+0000 7fb396000640 1 -- 172.21.15.105:0/2118476028 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb388075b20 con 0x7fb3a00acdb0 2024-03-23T07:58:24.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.370+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3a0073390 con 0x7fb3a00acdb0 2024-03-23T07:58:24.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.371+0000 7fb396000640 1 -- 172.21.15.105:0/2118476028 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb388046070 con 0x7fb3a00acdb0 2024-03-23T07:58:24.371 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.373+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb37c06ac00 msgr2=0x7fb37c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.373+0000 7fb3a4e00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb37c06ac00 0x7fb37c06d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb390007520 tx=0x7fb39000b040 comp rx=0 tx=0).stop 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.373+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3a00acdb0 msgr2=0x7fb3a015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.373+0000 7fb3a4e00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3a00acdb0 0x7fb3a015b9a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb38802c450 tx=0x7fb388001a00 comp rx=0 tx=0).stop 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 shutdown_connections 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3a00b1390 0x7fb3a00b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3a00acdb0 0x7fb3a015b9a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fb37c06ac00 0x7fb37c06d0c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 --2- 172.21.15.105:0/2118476028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb384003f20 0x7fb3a01f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 >> 172.21.15.105:0/2118476028 conn(0x7fb3a006a0b0 msgr2=0x7fb3a01544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 shutdown_connections 2024-03-23T07:58:24.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.374+0000 7fb3a4e00640 1 -- 172.21.15.105:0/2118476028 wait complete. 2024-03-23T07:58:24.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:24.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.599+0000 7f0b2d200640 1 Processor -- start 2024-03-23T07:58:24.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.599+0000 7f0b2d200640 1 -- start start 2024-03-23T07:58:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b2d200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b2815cce0 0x7f0b280cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b2d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0b280b4350 0x7f0b280b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b2d200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b280b4ce0 con 0x7f0b280b1740 2024-03-23T07:58:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b2d200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2815d0c0 con 0x7f0b2815cce0 2024-03-23T07:58:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b2d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280d0ae0 con 0x7f0b280b4350 2024-03-23T07:58:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b25e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b2815cce0 0x7f0b280cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b25e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b2815cce0 0x7f0b280cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54838/0 (socket says 172.21.15.105:54838) 2024-03-23T07:58:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b1fe00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0b280b1740 0x7f0b280b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49414/0 (socket says 172.21.15.105:49414) 2024-03-23T07:58:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b25e00640 1 -- 172.21.15.105:0/64819493 learned_addr learned my addr 172.21.15.105:0/64819493 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:24.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.601+0000 7f0b26800640 1 --2- 172.21.15.105:0/64819493 >> v2:172.21.15.132:3300/0 conn(0x7f0b280b4350 0x7f0b280b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.602+0000 7f0b25e00640 1 -- 172.21.15.105:0/64819493 >> v2:172.21.15.132:3300/0 conn(0x7f0b280b4350 msgr2=0x7f0b280b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:24.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.602+0000 7f0b25e00640 1 --2- 172.21.15.105:0/64819493 >> v2:172.21.15.132:3300/0 conn(0x7f0b280b4350 0x7f0b280b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.602+0000 7f0b25e00640 1 -- 172.21.15.105:0/64819493 >> v1:172.21.15.105:6789/0 conn(0x7f0b280b1740 legacy=0x7f0b280b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.602+0000 7f0b25400640 1 -- 172.21.15.105:0/64819493 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546214054 0 0) 0x7f0b10001110 con 0x7f0b280b1740 2024-03-23T07:58:24.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.602+0000 7f0b25e00640 1 -- 172.21.15.105:0/64819493 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b280d0d80 con 0x7f0b2815cce0 2024-03-23T07:58:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.603+0000 7f0b25e00640 1 --2- 172.21.15.105:0/64819493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b2815cce0 0x7f0b280cdaf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b280b8480 tx=0x7f0b1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ed4c3e249d123f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.603+0000 7f0b25400640 1 -- 172.21.15.105:0/64819493 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030860 con 0x7f0b2815cce0 2024-03-23T07:58:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.603+0000 7f0b25400640 1 -- 172.21.15.105:0/64819493 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b180309c0 con 0x7f0b2815cce0 2024-03-23T07:58:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.603+0000 7f0b25400640 1 -- 172.21.15.105:0/64819493 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030d50 con 0x7f0b2815cce0 2024-03-23T07:58:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 -- 172.21.15.105:0/64819493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b2815cce0 msgr2=0x7f0b280cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 --2- 172.21.15.105:0/64819493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b2815cce0 0x7f0b280cdaf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b280b8480 tx=0x7f0b1802b9a0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 -- 172.21.15.105:0/64819493 shutdown_connections 2024-03-23T07:58:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 --2- 172.21.15.105:0/64819493 >> v2:172.21.15.132:3300/0 conn(0x7f0b280b4350 0x7f0b280b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 --2- 172.21.15.105:0/64819493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b2815cce0 0x7f0b280cdaf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 -- 172.21.15.105:0/64819493 >> 172.21.15.105:0/64819493 conn(0x7f0b280cf350 msgr2=0x7f0b280cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 -- 172.21.15.105:0/64819493 shutdown_connections 2024-03-23T07:58:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.604+0000 7f0b2d200640 1 -- 172.21.15.105:0/64819493 wait complete. 2024-03-23T07:58:24.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.605+0000 7f0b2d200640 1 Processor -- start 2024-03-23T07:58:24.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.606+0000 7f0b2d200640 1 -- start start 2024-03-23T07:58:24.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.606+0000 7f0b2d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b280b1740 0x7f0b28150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b2d200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b280b4350 0x7f0b280d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b2d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2815cce0 0x7f0b280cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b2d200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2815fb10 con 0x7f0b280b4350 2024-03-23T07:58:24.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b2d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280b14c0 con 0x7f0b2815cce0 2024-03-23T07:58:24.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b2d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2815ee90 con 0x7f0b280b1740 2024-03-23T07:58:24.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b280b4350 0x7f0b280d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b280b4350 0x7f0b280d2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54854/0 (socket says 172.21.15.105:54854) 2024-03-23T07:58:24.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 -- 172.21.15.105:0/294588327 learned_addr learned my addr 172.21.15.105:0/294588327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b25e00640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b280b1740 0x7f0b28150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 -- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b280b1740 msgr2=0x7f0b28150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b280b1740 0x7f0b28150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b26800640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2815cce0 0x7f0b280cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 -- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2815cce0 msgr2=0x7f0b280cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2815cce0 0x7f0b280cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b1fe00640 1 -- 172.21.15.105:0/294588327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180094d0 con 0x7f0b280b4350 2024-03-23T07:58:24.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.607+0000 7f0b26800640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2815cce0 0x7f0b280cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.608+0000 7f0b1fe00640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b280b4350 0x7f0b280d2ee0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c780 tx=0x7f0b1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.608+0000 7f0b1e000640 1 -- 172.21.15.105:0/294588327 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100072d0 con 0x7f0b280b4350 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.608+0000 7f0b1e000640 1 -- 172.21.15.105:0/294588327 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10007450 con 0x7f0b280b4350 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.608+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2815f170 con 0x7f0b280b4350 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.608+0000 7f0b1e000640 1 -- 172.21.15.105:0/294588327 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100047e0 con 0x7f0b280b4350 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.608+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2815f730 con 0x7f0b280b4350 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.609+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2815fc50 con 0x7f0b280b4350 2024-03-23T07:58:24.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.610+0000 7f0b1e000640 1 -- 172.21.15.105:0/294588327 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b10004960 con 0x7f0b280b4350 2024-03-23T07:58:24.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.611+0000 7f0b1e000640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b0406ac00 0x7f0b0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:24.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.611+0000 7f0b1e000640 1 -- 172.21.15.105:0/294588327 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f0b10088650 con 0x7f0b280b4350 2024-03-23T07:58:24.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.611+0000 7f0b25e00640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b0406ac00 0x7f0b0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:24.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.612+0000 7f0b25e00640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b0406ac00 0x7f0b0406d0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b18036000 tx=0x7f0b1803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.615+0000 7f0b1e000640 1 -- 172.21.15.105:0/294588327 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b2815fc50 con 0x7f0b280b4350 2024-03-23T07:58:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.849+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b280b4ce0 con 0x7f0b280b4350 2024-03-23T07:58:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.850+0000 7f0b1e000640 1 -- 172.21.15.105:0/294588327 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f0b280b4ce0 con 0x7f0b280b4350 2024-03-23T07:58:24.851 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:24.851 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/12048571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":12048571},{"type":"v1","addr":"172.21.15.105:6837","nonce":12048571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:58:18.306711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5253":{"gid":5253,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 22 2024-03-23T07:58:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b0406ac00 msgr2=0x7f0b0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b0406ac00 0x7f0b0406d0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b18036000 tx=0x7f0b1803b040 comp rx=0 tx=0).stop 2024-03-23T07:58:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b280b4350 msgr2=0x7f0b280d2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b280b4350 0x7f0b280d2ee0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c780 tx=0x7f0b1000cc40 comp rx=0 tx=0).stop 2024-03-23T07:58:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 shutdown_connections 2024-03-23T07:58:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2815cce0 0x7f0b280cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b280b4350 0x7f0b280d2ee0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0b0406ac00 0x7f0b0406d0c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 --2- 172.21.15.105:0/294588327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b280b1740 0x7f0b28150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.853+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 >> 172.21.15.105:0/294588327 conn(0x7f0b280cf350 msgr2=0x7f0b280cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.854+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 shutdown_connections 2024-03-23T07:58:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:24.854+0000 7f0b2d200640 1 -- 172.21.15.105:0/294588327 wait complete. 2024-03-23T07:58:24.873 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:58:25.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.071+0000 7feee5a00640 1 Processor -- start 2024-03-23T07:58:25.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.071+0000 7feee5a00640 1 -- start start 2024-03-23T07:58:25.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feee5a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feee014aa40 0x7feee00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feee5a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feee00cc7b0 0x7feee00ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feee5a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee006da70 con 0x7feee00b27e0 2024-03-23T07:58:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feee5a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee00b1d00 con 0x7feee00cc7b0 2024-03-23T07:58:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feee5a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee00b1fe0 con 0x7feee014aa40 2024-03-23T07:58:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feede000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feee00cc7b0 0x7feee00ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feede000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feee00cc7b0 0x7feee00ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54858/0 (socket says 172.21.15.105:54858) 2024-03-23T07:58:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feede000640 1 -- 172.21.15.105:0/4277026640 learned_addr learned my addr 172.21.15.105:0/4277026640 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feede000640 1 -- 172.21.15.105:0/4277026640 >> v2:172.21.15.132:3300/0 conn(0x7feee014aa40 msgr2=0x7feee00cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feede000640 1 --2- 172.21.15.105:0/4277026640 >> v2:172.21.15.132:3300/0 conn(0x7feee014aa40 0x7feee00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feede000640 1 -- 172.21.15.105:0/4277026640 >> v1:172.21.15.132:6789/0 conn(0x7feee00b27e0 legacy=0x7feee00b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.073+0000 7feede000640 1 -- 172.21.15.105:0/4277026640 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00b22c0 con 0x7feee00cc7b0 2024-03-23T07:58:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.074+0000 7feede000640 1 --2- 172.21.15.105:0/4277026640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee00cc7b0 0x7feee00ccc60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7feee00b4960 tx=0x7feecc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eed0e9bff3a009 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feedd600640 1 -- 172.21.15.105:0/4277026640 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc031870 con 0x7feee00cc7b0 2024-03-23T07:58:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feedd600640 1 -- 172.21.15.105:0/4277026640 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc0319d0 con 0x7feee00cc7b0 2024-03-23T07:58:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feedd600640 1 -- 172.21.15.105:0/4277026640 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc031d60 con 0x7feee00cc7b0 2024-03-23T07:58:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feee5a00640 1 -- 172.21.15.105:0/4277026640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee00cc7b0 msgr2=0x7feee00ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feee5a00640 1 --2- 172.21.15.105:0/4277026640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee00cc7b0 0x7feee00ccc60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7feee00b4960 tx=0x7feecc02b910 comp rx=0 tx=0).stop 2024-03-23T07:58:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feee5a00640 1 -- 172.21.15.105:0/4277026640 shutdown_connections 2024-03-23T07:58:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feee5a00640 1 --2- 172.21.15.105:0/4277026640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee00cc7b0 0x7feee00ccc60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feee5a00640 1 --2- 172.21.15.105:0/4277026640 >> v2:172.21.15.132:3300/0 conn(0x7feee014aa40 0x7feee00cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feee5a00640 1 -- 172.21.15.105:0/4277026640 >> 172.21.15.105:0/4277026640 conn(0x7feee006a3d0 msgr2=0x7feee00b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:25.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.075+0000 7feee5a00640 1 -- 172.21.15.105:0/4277026640 shutdown_connections 2024-03-23T07:58:25.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.076+0000 7feee5a00640 1 -- 172.21.15.105:0/4277026640 wait complete. 2024-03-23T07:58:25.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.077+0000 7feee5a00640 1 Processor -- start 2024-03-23T07:58:25.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.077+0000 7feee5a00640 1 -- start start 2024-03-23T07:58:25.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.078+0000 7feee5a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee00b27e0 0x7feee0160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.078+0000 7feee5a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feee00cc7b0 0x7feee00a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.078+0000 7feee5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee014aa40 0x7feee00aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.078+0000 7feee5a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee0163180 con 0x7feee014aa40 2024-03-23T07:58:25.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.078+0000 7feee5a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee0163460 con 0x7feee00b27e0 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.078+0000 7feee5a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee0163740 con 0x7feee00cc7b0 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee014aa40 0x7feee00aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee014aa40 0x7feee00aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54872/0 (socket says 172.21.15.105:54872) 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 -- 172.21.15.105:0/3439585392 learned_addr learned my addr 172.21.15.105:0/3439585392 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 -- 172.21.15.105:0/3439585392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feee00cc7b0 msgr2=0x7feee00a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feee00cc7b0 0x7feee00a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 -- 172.21.15.105:0/3439585392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee00b27e0 msgr2=0x7feee0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee00b27e0 0x7feee0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 -- 172.21.15.105:0/3439585392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0094d0 con 0x7feee014aa40 2024-03-23T07:58:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feedf400640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee014aa40 0x7feee00aa4c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7feed800c780 tx=0x7feed800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.079+0000 7feed6a00640 1 -- 172.21.15.105:0/3439585392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8003d80 con 0x7feee014aa40 2024-03-23T07:58:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.080+0000 7feed6a00640 1 -- 172.21.15.105:0/3439585392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed800ce90 con 0x7feee014aa40 2024-03-23T07:58:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.080+0000 7feed6a00640 1 -- 172.21.15.105:0/3439585392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80052b0 con 0x7feee014aa40 2024-03-23T07:58:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.080+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee0163a20 con 0x7feee014aa40 2024-03-23T07:58:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.080+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee0144ce0 con 0x7feee014aa40 2024-03-23T07:58:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.081+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee0163f40 con 0x7feee014aa40 2024-03-23T07:58:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.082+0000 7feed6a00640 1 -- 172.21.15.105:0/3439585392 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feed8005410 con 0x7feee014aa40 2024-03-23T07:58:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.083+0000 7feed6a00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feec006ac30 0x7feec006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.083+0000 7feed6a00640 1 -- 172.21.15.105:0/3439585392 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7feed8088990 con 0x7feee014aa40 2024-03-23T07:58:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.083+0000 7feedea00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feec006ac30 0x7feec006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.084+0000 7feedea00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feec006ac30 0x7feec006d0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feed0009670 tx=0x7feed0006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.087+0000 7feed6a00640 1 -- 172.21.15.105:0/3439585392 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feee0163f40 con 0x7feee014aa40 2024-03-23T07:58:25.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.281+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feee0163f40 con 0x7feee014aa40 2024-03-23T07:58:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.282+0000 7feed6a00640 1 -- 172.21.15.105:0/3439585392 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16615 (secure 0 0 0) 0x7feee0163f40 con 0x7feee014aa40 2024-03-23T07:58:25.283 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":44,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:58:14.253586+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:57:51.204124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-23T07:57:52.349446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1e","mappings":[{"from":3,"to":5}]},{"pgid":"6.24","mappings":[{"from":4,"to":7}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]},{"pgid":"6.39","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/3723879365":"2024-03-24T07:58:14.253538+0000","172.21.15.105:6836/3723879365":"2024-03-24T07:58:14.253538+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:58:25.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feec006ac30 msgr2=0x7feec006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feec006ac30 0x7feec006d0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feed0009670 tx=0x7feed0006bf0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee014aa40 msgr2=0x7feee00aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee014aa40 0x7feee00aa4c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7feed800c780 tx=0x7feed800cc40 comp rx=0 tx=0).stop 2024-03-23T07:58:25.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 shutdown_connections 2024-03-23T07:58:25.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee014aa40 0x7feee00aa4c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feec006ac30 0x7feec006d0f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feee00cc7b0 0x7feee00a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 --2- 172.21.15.105:0/3439585392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee00b27e0 0x7feee0160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.285+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 >> 172.21.15.105:0/3439585392 conn(0x7feee006a3d0 msgr2=0x7feee00b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:25.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.286+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 shutdown_connections 2024-03-23T07:58:25.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.286+0000 7feee5a00640 1 -- 172.21.15.105:0/3439585392 wait complete. 2024-03-23T07:58:25.303 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T07:58:25.303 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:25.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.504+0000 7fbb7a200640 1 Processor -- start 2024-03-23T07:58:25.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.504+0000 7fbb7a200640 1 -- start start 2024-03-23T07:58:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb7a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb740b1db0 0x7fbb74167880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb7a200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb74161640 con 0x7fbb7413faa0 2024-03-23T07:58:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb7a200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740b38a0 con 0x7fbb74160800 2024-03-23T07:58:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb7a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740b3b80 con 0x7fbb740b1db0 2024-03-23T07:58:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb73400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbb7413faa0 0x7fbb74160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49426/0 (socket says 172.21.15.105:49426) 2024-03-23T07:58:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb73400640 1 -- 172.21.15.105:0/861236581 learned_addr learned my addr 172.21.15.105:0/861236581 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:25.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb73e00640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb740b1db0 0x7fbb74167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.506+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235962446 0 0) 0x7fbb74161640 con 0x7fbb7413faa0 2024-03-23T07:58:25.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb58003520 con 0x7fbb7413faa0 2024-03-23T07:58:25.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2866445827 0 0) 0x7fbb58003520 con 0x7fbb7413faa0 2024-03-23T07:58:25.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb740b1db0 msgr2=0x7fbb74167880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb740b1db0 0x7fbb74167880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 >> v1:172.21.15.132:6789/0 conn(0x7fbb74160800 legacy=0x7fbb740b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740b3e60 con 0x7fbb7413faa0 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fbb64001670 con 0x7fbb7413faa0 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 >> v1:172.21.15.105:6789/0 conn(0x7fbb7413faa0 legacy=0x7fbb74160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb580039a0 0x7fbb58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb580047c0 0x7fbb58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb740b1db0 0x7fbb58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740b3b80 con 0x7fbb580039a0 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5800ba60 con 0x7fbb580047c0 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5800bc20 con 0x7fbb740b1db0 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.507+0000 7fbb73400640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb580039a0 0x7fbb58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.508+0000 7fbb73400640 1 -- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb740b1db0 msgr2=0x7fbb58005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.508+0000 7fbb73400640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb740b1db0 0x7fbb58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.508+0000 7fbb73400640 1 -- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb580047c0 msgr2=0x7fbb58004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:25.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.508+0000 7fbb73400640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb580047c0 0x7fbb58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.508+0000 7fbb73400640 1 -- 172.21.15.105:0/861236581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58003520 con 0x7fbb580039a0 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.509+0000 7fbb73400640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb580039a0 0x7fbb58003dc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c210 tx=0x7fbb580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431ef5cfbebd87ed server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.509+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64005ad0 con 0x7fbb580039a0 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.509+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64005c30 con 0x7fbb580039a0 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.509+0000 7fbb72000640 1 -- 172.21.15.105:0/861236581 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6402c870 con 0x7fbb580039a0 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.510+0000 7fbb7a200640 1 -- 172.21.15.105:0/861236581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb580039a0 msgr2=0x7fbb58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.510+0000 7fbb7a200640 1 --2- 172.21.15.105:0/861236581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb580039a0 0x7fbb58003dc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c210 tx=0x7fbb580057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.510+0000 7fbb73e00640 1 -- 172.21.15.105:0/861236581 reap_dead start 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.510+0000 7fbb7a200640 1 -- 172.21.15.105:0/861236581 shutdown_connections 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.510+0000 7fbb7a200640 1 -- 172.21.15.105:0/861236581 >> 172.21.15.105:0/861236581 conn(0x7fbb7406a3d0 msgr2=0x7fbb740b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.510+0000 7fbb7a200640 1 -- 172.21.15.105:0/861236581 shutdown_connections 2024-03-23T07:58:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.510+0000 7fbb7a200640 1 -- 172.21.15.105:0/861236581 wait complete. 2024-03-23T07:58:25.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.511+0000 7fbb7a200640 1 Processor -- start 2024-03-23T07:58:25.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.512+0000 7fbb7a200640 1 -- start start 2024-03-23T07:58:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.512+0000 7fbb7a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb580039a0 0x7fbb74156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb7a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb740afb90 0x7fbb740b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb7a200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb74142280 0x7fbb74142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb7a200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740b0550 con 0x7fbb74142280 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb7a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740cb260 con 0x7fbb740afb90 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb7a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74143030 con 0x7fbb580039a0 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb74142280 0x7fbb74142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb74142280 0x7fbb74142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54902/0 (socket says 172.21.15.105:54902) 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 -- 172.21.15.105:0/965905176 learned_addr learned my addr 172.21.15.105:0/965905176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb72a00640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb740afb90 0x7fbb740b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73400640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb580039a0 0x7fbb74156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 -- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb580039a0 msgr2=0x7fbb74156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb580039a0 0x7fbb74156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 -- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb740afb90 msgr2=0x7fbb740b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb740afb90 0x7fbb740b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb73e00640 1 -- 172.21.15.105:0/965905176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740ca930 con 0x7fbb74142280 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.513+0000 7fbb72a00640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb740afb90 0x7fbb740b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.514+0000 7fbb73e00640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb74142280 0x7fbb74142700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbb74071440 tx=0x7fbb6c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.514+0000 7fbb6b400640 1 -- 172.21.15.105:0/965905176 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c010040 con 0x7fbb74142280 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.514+0000 7fbb6b400640 1 -- 172.21.15.105:0/965905176 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c00bae0 con 0x7fbb74142280 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.514+0000 7fbb6b400640 1 -- 172.21.15.105:0/965905176 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00be70 con 0x7fbb74142280 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.514+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb740cac10 con 0x7fbb74142280 2024-03-23T07:58:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.514+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb74157270 con 0x7fbb74142280 2024-03-23T07:58:25.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.515+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb74073390 con 0x7fbb74142280 2024-03-23T07:58:25.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.516+0000 7fbb6b400640 1 -- 172.21.15.105:0/965905176 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbb6c01e410 con 0x7fbb74142280 2024-03-23T07:58:25.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.517+0000 7fbb6b400640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb5406ac00 0x7fbb5406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.517+0000 7fbb6b400640 1 -- 172.21.15.105:0/965905176 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fbb6c089820 con 0x7fbb74142280 2024-03-23T07:58:25.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.517+0000 7fbb73400640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb5406ac00 0x7fbb5406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.518+0000 7fbb73400640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb5406ac00 0x7fbb5406d0c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb64034000 tx=0x7fbb64009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.522+0000 7fbb6b400640 1 -- 172.21.15.105:0/965905176 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbb6c0581e0 con 0x7fbb74142280 2024-03-23T07:58:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.753+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb74066d50 con 0x7fbb74142280 2024-03-23T07:58:25.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.755+0000 7fbb6b400640 1 -- 172.21.15.105:0/965905176 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fbb6c057d70 con 0x7fbb74142280 2024-03-23T07:58:25.756 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:25.756 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/12048571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":12048571},{"type":"v1","addr":"172.21.15.105:6837","nonce":12048571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:58:18.306711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5253":{"gid":5253,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:25.756 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 22 2024-03-23T07:58:25.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb5406ac00 msgr2=0x7fbb5406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb5406ac00 0x7fbb5406d0c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb64034000 tx=0x7fbb64009030 comp rx=0 tx=0).stop 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb74142280 msgr2=0x7fbb74142700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb74142280 0x7fbb74142700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbb74071440 tx=0x7fbb6c00b480 comp rx=0 tx=0).stop 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 shutdown_connections 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb74142280 0x7fbb74142700 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb740afb90 0x7fbb740b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fbb5406ac00 0x7fbb5406d0c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 --2- 172.21.15.105:0/965905176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb580039a0 0x7fbb74156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 >> 172.21.15.105:0/965905176 conn(0x7fbb7406a3d0 msgr2=0x7fbb7413f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 shutdown_connections 2024-03-23T07:58:25.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.758+0000 7fbb7a200640 1 -- 172.21.15.105:0/965905176 wait complete. 2024-03-23T07:58:25.777 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:58:25.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.984+0000 7f36c9c00640 1 Processor -- start 2024-03-23T07:58:25.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.985+0000 7f36c9c00640 1 -- start start 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.986+0000 7f36c9c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f36c40adbc0 0x7f36c40b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.986+0000 7f36c9c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36c406d390 0x7f36c406d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.986+0000 7f36c9c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40b16a0 con 0x7f36c40cbab0 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.986+0000 7f36c9c00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c406dd40 con 0x7f36c406d390 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.986+0000 7f36c9c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c40b4700 con 0x7f36c40adbc0 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.986+0000 7f36c3400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36c406d390 0x7f36c406d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c3400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36c406d390 0x7f36c406d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54904/0 (socket says 172.21.15.105:54904) 2024-03-23T07:58:25.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36ba000640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f36c40cbab0 0x7f36c40b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49440/0 (socket says 172.21.15.105:49440) 2024-03-23T07:58:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c3400640 1 -- 172.21.15.105:0/4222528263 learned_addr learned my addr 172.21.15.105:0/4222528263 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c2a00640 1 --2- 172.21.15.105:0/4222528263 >> v2:172.21.15.132:3300/0 conn(0x7f36c40adbc0 0x7f36c40b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c3400640 1 -- 172.21.15.105:0/4222528263 >> v2:172.21.15.132:3300/0 conn(0x7f36c40adbc0 msgr2=0x7f36c40b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c3400640 1 --2- 172.21.15.105:0/4222528263 >> v2:172.21.15.132:3300/0 conn(0x7f36c40adbc0 0x7f36c40b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c3400640 1 -- 172.21.15.105:0/4222528263 >> v1:172.21.15.105:6789/0 conn(0x7f36c40cbab0 legacy=0x7f36c40b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c3400640 1 -- 172.21.15.105:0/4222528263 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40cbfb0 con 0x7f36c406d390 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.987+0000 7f36c2000640 1 -- 172.21.15.105:0/4222528263 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805962061 0 0) 0x7f36c40b16a0 con 0x7f36c40cbab0 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.988+0000 7f36c3400640 1 --2- 172.21.15.105:0/4222528263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c406d390 0x7f36c406d800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f36c406e020 tx=0x7f36bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4882efabf67f9cb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.988+0000 7f36c2000640 1 -- 172.21.15.105:0/4222528263 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc02ca80 con 0x7f36c406d390 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.988+0000 7f36c2000640 1 -- 172.21.15.105:0/4222528263 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36bc02cbe0 con 0x7f36c406d390 2024-03-23T07:58:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.988+0000 7f36c2000640 1 -- 172.21.15.105:0/4222528263 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0343f0 con 0x7f36c406d390 2024-03-23T07:58:25.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 -- 172.21.15.105:0/4222528263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c406d390 msgr2=0x7f36c406d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 --2- 172.21.15.105:0/4222528263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c406d390 0x7f36c406d800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f36c406e020 tx=0x7f36bc02bee0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 -- 172.21.15.105:0/4222528263 shutdown_connections 2024-03-23T07:58:25.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 --2- 172.21.15.105:0/4222528263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c406d390 0x7f36c406d800 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 --2- 172.21.15.105:0/4222528263 >> v2:172.21.15.132:3300/0 conn(0x7f36c40adbc0 0x7f36c40b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 -- 172.21.15.105:0/4222528263 >> 172.21.15.105:0/4222528263 conn(0x7f36c406a3d0 msgr2=0x7f36c40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:25.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 -- 172.21.15.105:0/4222528263 shutdown_connections 2024-03-23T07:58:25.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.989+0000 7f36c9c00640 1 -- 172.21.15.105:0/4222528263 wait complete. 2024-03-23T07:58:25.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.991+0000 7f36c9c00640 1 Processor -- start 2024-03-23T07:58:25.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.991+0000 7f36c9c00640 1 -- start start 2024-03-23T07:58:25.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c9c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36c406d390 0x7f36c413b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c9c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40adbc0 0x7f36c4138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c9c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c40cbab0 0x7f36c4158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c9c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4133320 con 0x7f36c40cbab0 2024-03-23T07:58:25.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c9c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4139960 con 0x7f36c40adbc0 2024-03-23T07:58:25.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c9c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4139c40 con 0x7f36c406d390 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c40cbab0 0x7f36c4158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c40cbab0 0x7f36c4158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54908/0 (socket says 172.21.15.105:54908) 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.992+0000 7f36c3400640 1 -- 172.21.15.105:0/1185548080 learned_addr learned my addr 172.21.15.105:0/1185548080 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c3400640 1 -- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36c406d390 msgr2=0x7f36c413b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36ba000640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40adbc0 0x7f36c4138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c2a00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36c406d390 0x7f36c413b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c3400640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36c406d390 0x7f36c413b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c3400640 1 -- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40adbc0 msgr2=0x7f36c4138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c3400640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40adbc0 0x7f36c4138ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c3400640 1 -- 172.21.15.105:0/1185548080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc009b30 con 0x7f36c40cbab0 2024-03-23T07:58:25.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c3400640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c40cbab0 0x7f36c4158fd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f36bc036040 tx=0x7f36bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:25.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36bb400640 1 -- 172.21.15.105:0/1185548080 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc009510 con 0x7f36c40cbab0 2024-03-23T07:58:25.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36bb400640 1 -- 172.21.15.105:0/1185548080 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36bc03e450 con 0x7f36c40cbab0 2024-03-23T07:58:25.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c4139f20 con 0x7f36c40cbab0 2024-03-23T07:58:25.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36bb400640 1 -- 172.21.15.105:0/1185548080 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc03e7e0 con 0x7f36c40cbab0 2024-03-23T07:58:25.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.993+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c413b590 con 0x7f36c40cbab0 2024-03-23T07:58:25.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.994+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c406ac20 con 0x7f36c40cbab0 2024-03-23T07:58:25.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.995+0000 7f36bb400640 1 -- 172.21.15.105:0/1185548080 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f36bc02cd50 con 0x7f36c40cbab0 2024-03-23T07:58:25.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.996+0000 7f36bb400640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36a006ac00 0x7f36a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:25.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.996+0000 7f36bb400640 1 -- 172.21.15.105:0/1185548080 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f36bc0ac110 con 0x7f36c40cbab0 2024-03-23T07:58:25.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.996+0000 7f36c2a00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36a006ac00 0x7f36a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:25.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:25.997+0000 7f36c2a00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36a006ac00 0x7f36a006d0c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f36b4006fd0 tx=0x7f36b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:26.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.000+0000 7f36bb400640 1 -- 172.21.15.105:0/1185548080 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f36bc077160 con 0x7f36c40cbab0 2024-03-23T07:58:26.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.234+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c4073390 con 0x7f36c40cbab0 2024-03-23T07:58:26.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.235+0000 7f36bb400640 1 -- 172.21.15.105:0/1185548080 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f36bc07a780 con 0x7f36c40cbab0 2024-03-23T07:58:26.236 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:26.236 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/12048571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":12048571},{"type":"v1","addr":"172.21.15.105:6837","nonce":12048571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T07:57:53.325549+0000","modified":"2024-03-23T07:58:18.306711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5253":{"gid":5253,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6835/2670581804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2670581804},{"type":"v1","addr":"172.21.15.105:6835","nonce":2670581804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-23T07:58:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 22 2024-03-23T07:58:26.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36a006ac00 msgr2=0x7f36a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36a006ac00 0x7f36a006d0c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f36b4006fd0 tx=0x7f36b4008040 comp rx=0 tx=0).stop 2024-03-23T07:58:26.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c40cbab0 msgr2=0x7f36c4158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c40cbab0 0x7f36c4158fd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f36bc036040 tx=0x7f36bc004a40 comp rx=0 tx=0).stop 2024-03-23T07:58:26.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 shutdown_connections 2024-03-23T07:58:26.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36c40cbab0 0x7f36c4158fd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40adbc0 0x7f36c4138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f36a006ac00 0x7f36a006d0c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 --2- 172.21.15.105:0/1185548080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36c406d390 0x7f36c413b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.237+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 >> 172.21.15.105:0/1185548080 conn(0x7f36c406a3d0 msgr2=0x7f36c40b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:26.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.238+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 shutdown_connections 2024-03-23T07:58:26.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.238+0000 7f36c9c00640 1 -- 172.21.15.105:0/1185548080 wait complete. 2024-03-23T07:58:26.258 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:58:26.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.459+0000 7f0c84600640 1 Processor -- start 2024-03-23T07:58:26.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.459+0000 7f0c84600640 1 -- start start 2024-03-23T07:58:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.460+0000 7f0c84600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 0x7f0c7c167880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.460+0000 7f0c84600640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c161640 con 0x7f0c7c160800 2024-03-23T07:58:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.460+0000 7f0c84600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c0b38a0 con 0x7f0c7c13faa0 2024-03-23T07:58:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.460+0000 7f0c84600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0b3b80 con 0x7f0c7c0b1db0 2024-03-23T07:58:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.461+0000 7f0c81200640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0c7c160800 0x7f0c7c0b16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49456/0 (socket says 172.21.15.105:49456) 2024-03-23T07:58:26.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.461+0000 7f0c81200640 1 -- 172.21.15.105:0/924434483 learned_addr learned my addr 172.21.15.105:0/924434483 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:26.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.461+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988561463 0 0) 0x7f0c7c161640 con 0x7f0c7c160800 2024-03-23T07:58:26.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.461+0000 7f0c82600640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 0x7f0c7c167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.461+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c64003520 con 0x7f0c7c160800 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1985866537 0 0) 0x7f0c64003520 con 0x7f0c7c160800 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 msgr2=0x7f0c7c167880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 0x7f0c7c167880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 >> v1:172.21.15.132:6789/0 conn(0x7f0c7c13faa0 legacy=0x7f0c7c160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0b3e60 con 0x7f0c7c160800 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f0c70001670 con 0x7f0c7c160800 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c82600640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 0x7f0c7c167880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 >> v1:172.21.15.105:6789/0 conn(0x7f0c7c160800 legacy=0x7f0c7c0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 0x7f0c640039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c64004d00 0x7f0c64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0b3b80 con 0x7f0c64003f90 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6400bae0 con 0x7f0c64004d00 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6400bca0 con 0x7f0c7c0b1db0 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c81200640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c81200640 1 -- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 msgr2=0x7f0c640039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c81c00640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 0x7f0c640039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c82600640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c64004d00 0x7f0c64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.462+0000 7f0c81200640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c0b1db0 0x7f0c640039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.463+0000 7f0c81200640 1 -- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c64004d00 msgr2=0x7f0c64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.463+0000 7f0c81200640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c64004d00 0x7f0c64005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.463+0000 7f0c81200640 1 -- 172.21.15.105:0/924434483 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64003520 con 0x7f0c64003f90 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.464+0000 7f0c81200640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c640047c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c240 tx=0x7f0c640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1e913abb28f297 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.464+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70005ad0 con 0x7f0c64003f90 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.464+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70005c30 con 0x7f0c64003f90 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.464+0000 7f0c7be00640 1 -- 172.21.15.105:0/924434483 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7002ca50 con 0x7f0c64003f90 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.465+0000 7f0c84600640 1 -- 172.21.15.105:0/924434483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 msgr2=0x7f0c640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.465+0000 7f0c84600640 1 --2- 172.21.15.105:0/924434483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c640047c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c240 tx=0x7f0c640057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.465+0000 7f0c82600640 1 -- 172.21.15.105:0/924434483 reap_dead start 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.465+0000 7f0c84600640 1 -- 172.21.15.105:0/924434483 shutdown_connections 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.465+0000 7f0c84600640 1 -- 172.21.15.105:0/924434483 >> 172.21.15.105:0/924434483 conn(0x7f0c7c06a3d0 msgr2=0x7f0c7c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.465+0000 7f0c84600640 1 -- 172.21.15.105:0/924434483 shutdown_connections 2024-03-23T07:58:26.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.465+0000 7f0c84600640 1 -- 172.21.15.105:0/924434483 wait complete. 2024-03-23T07:58:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.466+0000 7f0c84600640 1 Processor -- start 2024-03-23T07:58:26.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.467+0000 7f0c84600640 1 -- start start 2024-03-23T07:58:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.467+0000 7f0c84600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c7c142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.467+0000 7f0c84600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c1f6830 0x7f0c7c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.467+0000 7f0c84600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c7c146770 0x7f0c7c1f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.467+0000 7f0c84600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c134930 con 0x7f0c64003f90 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.467+0000 7f0c84600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0ac530 con 0x7f0c7c146770 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.467+0000 7f0c84600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c142e80 con 0x7f0c7c1f6830 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c7c142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c7c142940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54926/0 (socket says 172.21.15.105:54926) 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 -- 172.21.15.105:0/3907047454 learned_addr learned my addr 172.21.15.105:0/3907047454 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 -- 172.21.15.105:0/3907047454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c1f6830 msgr2=0x7f0c7c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c1f6830 0x7f0c7c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 -- 172.21.15.105:0/3907047454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c7c146770 msgr2=0x7f0c7c1f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c7c146770 0x7f0c7c1f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 -- 172.21.15.105:0/3907047454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0ab920 con 0x7f0c64003f90 2024-03-23T07:58:26.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.468+0000 7f0c81c00640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c7c142940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d570 tx=0x7f0c6800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.469+0000 7f0c79600640 1 -- 172.21.15.105:0/3907047454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68005480 con 0x7f0c64003f90 2024-03-23T07:58:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.469+0000 7f0c79600640 1 -- 172.21.15.105:0/3907047454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c68003610 con 0x7f0c64003f90 2024-03-23T07:58:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.469+0000 7f0c79600640 1 -- 172.21.15.105:0/3907047454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68003980 con 0x7f0c64003f90 2024-03-23T07:58:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.469+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c0abc00 con 0x7f0c64003f90 2024-03-23T07:58:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.469+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c0ac1c0 con 0x7f0c64003f90 2024-03-23T07:58:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.470+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c073390 con 0x7f0c64003f90 2024-03-23T07:58:26.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.471+0000 7f0c79600640 1 -- 172.21.15.105:0/3907047454 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0c68003ae0 con 0x7f0c64003f90 2024-03-23T07:58:26.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.472+0000 7f0c79600640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0c5c06ac00 0x7f0c5c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.472+0000 7f0c79600640 1 -- 172.21.15.105:0/3907047454 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f0c680894e0 con 0x7f0c64003f90 2024-03-23T07:58:26.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.472+0000 7f0c81200640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0c5c06ac00 0x7f0c5c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.473+0000 7f0c81200640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0c5c06ac00 0x7f0c5c06d0c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0c70034000 tx=0x7f0c70007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:26.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.477+0000 7f0c79600640 1 -- 172.21.15.105:0/3907047454 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0c68054410 con 0x7f0c64003f90 2024-03-23T07:58:26.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.671+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c7c0ac350 con 0x7f0c64003f90 2024-03-23T07:58:26.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.674+0000 7f0c79600640 1 -- 172.21.15.105:0/3907047454 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f0c68001230 con 0x7f0c64003f90 2024-03-23T07:58:26.675 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:26.675 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":44,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:58:14.253586+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-23T07:57:51.204124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-23T07:57:52.349446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1e","mappings":[{"from":3,"to":5}]},{"pgid":"6.24","mappings":[{"from":4,"to":7}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]},{"pgid":"6.39","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/3723879365":"2024-03-24T07:58:14.253538+0000","172.21.15.105:6836/3723879365":"2024-03-24T07:58:14.253538+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0c5c06ac00 msgr2=0x7f0c5c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0c5c06ac00 0x7f0c5c06d0c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0c70034000 tx=0x7f0c70007000 comp rx=0 tx=0).stop 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 msgr2=0x7f0c7c142940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c7c142940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d570 tx=0x7f0c6800da30 comp rx=0 tx=0).stop 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 shutdown_connections 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c7c146770 0x7f0c7c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f0c5c06ac00 0x7f0c5c06d0c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0c7c1f6830 0x7f0c7c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 --2- 172.21.15.105:0/3907047454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c64003f90 0x7f0c7c142940 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 >> 172.21.15.105:0/3907047454 conn(0x7f0c7c06a3d0 msgr2=0x7f0c7c0ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:26.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 shutdown_connections 2024-03-23T07:58:26.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.677+0000 7f0c84600640 1 -- 172.21.15.105:0/3907047454 wait complete. 2024-03-23T07:58:26.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T07:58:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.896+0000 7f484fe00640 1 Processor -- start 2024-03-23T07:58:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.897+0000 7f484fe00640 1 -- start start 2024-03-23T07:58:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.898+0000 7f484fe00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48500cad70 0x7f48500cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.898+0000 7f484fe00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500b25b0 con 0x7f4850140cf0 2024-03-23T07:58:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.898+0000 7f484fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48501410d0 con 0x7f4850141410 2024-03-23T07:58:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.898+0000 7f484fe00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48500cb720 con 0x7f48500cad70 2024-03-23T07:58:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.898+0000 7f484e000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48500cad70 0x7f48500cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.899+0000 7f484e000640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48500cad70 0x7f48500cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43562/0 (socket says 172.21.15.105:43562) 2024-03-23T07:58:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.899+0000 7f484e000640 1 -- 172.21.15.105:0/3753593924 learned_addr learned my addr 172.21.15.105:0/3753593924 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.899+0000 7f484e000640 1 -- 172.21.15.105:0/3753593924 >> v1:172.21.15.132:6789/0 conn(0x7f4850141410 legacy=0x7f4850141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.899+0000 7f484e000640 1 -- 172.21.15.105:0/3753593924 >> v1:172.21.15.105:6789/0 conn(0x7f4850140cf0 legacy=0x7f48500b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.899+0000 7f484e000640 1 -- 172.21.15.105:0/3753593924 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48500cd1b0 con 0x7f48500cad70 2024-03-23T07:58:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.900+0000 7f484e000640 1 --2- 172.21.15.105:0/3753593924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48500cad70 0x7f48500cb1e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f483c009e50 tx=0x7f483c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c089a88e1c629e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.900+0000 7f484d600640 1 -- 172.21.15.105:0/3753593924 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c02cb20 con 0x7f48500cad70 2024-03-23T07:58:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.900+0000 7f484d600640 1 -- 172.21.15.105:0/3753593924 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c02cc80 con 0x7f48500cad70 2024-03-23T07:58:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.900+0000 7f484d600640 1 -- 172.21.15.105:0/3753593924 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c0343f0 con 0x7f48500cad70 2024-03-23T07:58:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.901+0000 7f484fe00640 1 -- 172.21.15.105:0/3753593924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48500cad70 msgr2=0x7f48500cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.901+0000 7f484fe00640 1 --2- 172.21.15.105:0/3753593924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48500cad70 0x7f48500cb1e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f483c009e50 tx=0x7f483c02bf80 comp rx=0 tx=0).stop 2024-03-23T07:58:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.901+0000 7f484fe00640 1 -- 172.21.15.105:0/3753593924 shutdown_connections 2024-03-23T07:58:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.901+0000 7f484fe00640 1 --2- 172.21.15.105:0/3753593924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48500cad70 0x7f48500cb1e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.901+0000 7f484fe00640 1 -- 172.21.15.105:0/3753593924 >> 172.21.15.105:0/3753593924 conn(0x7f4850065e20 msgr2=0x7f4850069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.901+0000 7f484fe00640 1 -- 172.21.15.105:0/3753593924 shutdown_connections 2024-03-23T07:58:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.901+0000 7f484fe00640 1 -- 172.21.15.105:0/3753593924 wait complete. 2024-03-23T07:58:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.903+0000 7f484fe00640 1 Processor -- start 2024-03-23T07:58:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.903+0000 7f484fe00640 1 -- start start 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48500cad70 0x7f48501e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4850140cf0 0x7f485015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4850141410 0x7f48500c3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48501e3ea0 con 0x7f4850141410 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485013f610 con 0x7f48500cad70 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48501e3c90 con 0x7f4850140cf0 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4850141410 0x7f48500c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4850141410 0x7f48500c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43564/0 (socket says 172.21.15.105:43564) 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.904+0000 7f484f400640 1 -- 172.21.15.105:0/2998196970 learned_addr learned my addr 172.21.15.105:0/2998196970 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484f400640 1 -- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4850140cf0 msgr2=0x7f485015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484f400640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4850140cf0 0x7f485015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484f400640 1 -- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48500cad70 msgr2=0x7f48501e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484ea00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48500cad70 0x7f48501e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484f400640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48500cad70 0x7f48501e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484f400640 1 -- 172.21.15.105:0/2998196970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c009b30 con 0x7f4850141410 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484ea00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48500cad70 0x7f48501e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484f400640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4850141410 0x7f48500c3300 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f484800c410 tx=0x7f484800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f4847400640 1 -- 172.21.15.105:0/2998196970 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011790 con 0x7f4850141410 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f4847400640 1 -- 172.21.15.105:0/2998196970 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48480118f0 con 0x7f4850141410 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f4847400640 1 -- 172.21.15.105:0/2998196970 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011ce0 con 0x7f4850141410 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485015a710 con 0x7f4850141410 2024-03-23T07:58:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.905+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485015b1b0 con 0x7f4850141410 2024-03-23T07:58:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.907+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485004e030 con 0x7f4850141410 2024-03-23T07:58:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.907+0000 7f4847400640 1 -- 172.21.15.105:0/2998196970 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f484801c460 con 0x7f4850141410 2024-03-23T07:58:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.909+0000 7f4847400640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f483006ac00 0x7f483006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.909+0000 7f4847400640 1 -- 172.21.15.105:0/2998196970 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f4848089220 con 0x7f4850141410 2024-03-23T07:58:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.909+0000 7f484ea00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f483006ac00 0x7f483006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.910+0000 7f484ea00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f483006ac00 0x7f483006d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4840000f60 tx=0x7f4840009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:26.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:26.914+0000 7f4847400640 1 -- 172.21.15.105:0/2998196970 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f48480541e0 con 0x7f4850141410 2024-03-23T07:58:27.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.147+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4850143ce0 con 0x7f4850141410 2024-03-23T07:58:27.149 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-03-23T07:58:27.148+0000 7f31aa200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T07:58:27.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.164+0000 7f4847400640 1 -- 172.21.15.105:0/2998196970 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4848057800 con 0x7f4850141410 2024-03-23T07:58:27.165 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T07:58:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.167+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f483006ac00 msgr2=0x7f483006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.167+0000 7f484fe00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f483006ac00 0x7f483006d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4840000f60 tx=0x7f4840009040 comp rx=0 tx=0).stop 2024-03-23T07:58:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.167+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4850141410 msgr2=0x7f48500c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.167+0000 7f484fe00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4850141410 0x7f48500c3300 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f484800c410 tx=0x7f484800c8d0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 shutdown_connections 2024-03-23T07:58:27.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4850141410 0x7f48500c3300 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f483006ac00 0x7f483006d0c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4850140cf0 0x7f485015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 --2- 172.21.15.105:0/2998196970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48500cad70 0x7f48501e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 >> 172.21.15.105:0/2998196970 conn(0x7f4850065e20 msgr2=0x7f48500b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:27.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 shutdown_connections 2024-03-23T07:58:27.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.168+0000 7f484fe00640 1 -- 172.21.15.105:0/2998196970 wait complete. 2024-03-23T07:58:27.176 INFO:tasks.ceph.mds.c.smithi105.stderr: -366> 2024-03-23T07:58:20.528+0000 7fdf35000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T07:58:27.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T07:58:27.225 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-03-23T07:58:27.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.371+0000 7fe45f400640 1 Processor -- start 2024-03-23T07:58:27.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.371+0000 7fe45f400640 1 -- start start 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.372+0000 7fe45f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580cd740 0x7fe4580b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.372+0000 7fe45f400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe4580b5aa0 0x7fe4580b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.372+0000 7fe45f400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580d20c0 con 0x7fe4580d0d40 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.372+0000 7fe45f400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580d14b0 con 0x7fe4580b5aa0 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.372+0000 7fe45f400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580d1790 con 0x7fe4580cd740 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45d400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe4580b5aa0 0x7fe4580b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe457e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe4580d0d40 0x7fe4580cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43268/0 (socket says 172.21.15.105:43268) 2024-03-23T07:58:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe457e00640 1 -- 172.21.15.105:0/2994690991 learned_addr learned my addr 172.21.15.105:0/2994690991 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45d400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe4580b5aa0 0x7fe4580b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43588/0 (socket says 172.21.15.105:43588) 2024-03-23T07:58:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45ca00640 1 --2- 172.21.15.105:0/2994690991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580cd740 0x7fe4580b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45d400640 1 -- 172.21.15.105:0/2994690991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580cd740 msgr2=0x7fe4580b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45d400640 1 --2- 172.21.15.105:0/2994690991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580cd740 0x7fe4580b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe457400640 1 -- 172.21.15.105:0/2994690991 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173995 0 0) 0x7fe4580d20c0 con 0x7fe4580d0d40 2024-03-23T07:58:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45d400640 1 -- 172.21.15.105:0/2994690991 >> v1:172.21.15.105:6789/0 conn(0x7fe4580d0d40 legacy=0x7fe4580cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45d400640 1 -- 172.21.15.105:0/2994690991 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580d1a70 con 0x7fe4580b5aa0 2024-03-23T07:58:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.373+0000 7fe45ca00640 1 --2- 172.21.15.105:0/2994690991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580cd740 0x7fe4580b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.374+0000 7fe45d400640 1 --2- 172.21.15.105:0/2994690991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580b5aa0 0x7fe4580b5f30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe4480097a0 tx=0x7fe44802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70780642ede7d3fa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe457400640 1 -- 172.21.15.105:0/2994690991 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448031830 con 0x7fe4580b5aa0 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe457400640 1 -- 172.21.15.105:0/2994690991 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe448031990 con 0x7fe4580b5aa0 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe457400640 1 -- 172.21.15.105:0/2994690991 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448031d20 con 0x7fe4580b5aa0 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe45f400640 1 -- 172.21.15.105:0/2994690991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580b5aa0 msgr2=0x7fe4580b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe45f400640 1 --2- 172.21.15.105:0/2994690991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580b5aa0 0x7fe4580b5f30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe4480097a0 tx=0x7fe44802b8d0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe45f400640 1 -- 172.21.15.105:0/2994690991 shutdown_connections 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe45f400640 1 --2- 172.21.15.105:0/2994690991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580b5aa0 0x7fe4580b5f30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe45f400640 1 --2- 172.21.15.105:0/2994690991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580cd740 0x7fe4580b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.375+0000 7fe45f400640 1 -- 172.21.15.105:0/2994690991 >> 172.21.15.105:0/2994690991 conn(0x7fe4580cf5d0 msgr2=0x7fe4580cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.376+0000 7fe45f400640 1 -- 172.21.15.105:0/2994690991 shutdown_connections 2024-03-23T07:58:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.376+0000 7fe45f400640 1 -- 172.21.15.105:0/2994690991 wait complete. 2024-03-23T07:58:27.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.377+0000 7fe45f400640 1 Processor -- start 2024-03-23T07:58:27.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.377+0000 7fe45f400640 1 -- start start 2024-03-23T07:58:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.378+0000 7fe45f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580b5aa0 0x7fe4581f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.378+0000 7fe45f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580cd740 0x7fe4580caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.378+0000 7fe45f400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4580d0d40 0x7fe4580b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe45f400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4581582c0 con 0x7fe4580cd740 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe45f400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580cb2e0 con 0x7fe4580d0d40 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe45f400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580cb5c0 con 0x7fe4580b5aa0 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580cd740 0x7fe4580caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580cd740 0x7fe4580caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43602/0 (socket says 172.21.15.105:43602) 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 -- 172.21.15.105:0/3951114579 learned_addr learned my addr 172.21.15.105:0/3951114579 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe45ca00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580b5aa0 0x7fe4581f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 -- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580b5aa0 msgr2=0x7fe4581f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580b5aa0 0x7fe4581f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 -- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4580d0d40 msgr2=0x7fe4580b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4580d0d40 0x7fe4580b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe457e00640 1 -- 172.21.15.105:0/3951114579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448009400 con 0x7fe4580cd740 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.379+0000 7fe45ca00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580b5aa0 0x7fe4581f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.380+0000 7fe457e00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580cd740 0x7fe4580caa90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c430 tx=0x7fe44400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.380+0000 7fe454c00640 1 -- 172.21.15.105:0/3951114579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4440074e0 con 0x7fe4580cd740 2024-03-23T07:58:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.380+0000 7fe454c00640 1 -- 172.21.15.105:0/3951114579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe444007660 con 0x7fe4580cd740 2024-03-23T07:58:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.380+0000 7fe454c00640 1 -- 172.21.15.105:0/3951114579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444004a20 con 0x7fe4580cd740 2024-03-23T07:58:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.380+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4580cb8a0 con 0x7fe4580cd740 2024-03-23T07:58:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.380+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4581f37a0 con 0x7fe4580cd740 2024-03-23T07:58:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.381+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe458158750 con 0x7fe4580cd740 2024-03-23T07:58:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.382+0000 7fe454c00640 1 -- 172.21.15.105:0/3951114579 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe444004b80 con 0x7fe4580cd740 2024-03-23T07:58:27.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.383+0000 7fe454c00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe43806ac00 0x7fe43806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:27.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.383+0000 7fe454c00640 1 -- 172.21.15.105:0/3951114579 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fe444089450 con 0x7fe4580cd740 2024-03-23T07:58:27.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.383+0000 7fe45ca00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe43806ac00 0x7fe43806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:27.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.384+0000 7fe45ca00640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe43806ac00 0x7fe43806d0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe45000d430 tx=0x7fe450007530 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:27.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.388+0000 7fe454c00640 1 -- 172.21.15.105:0/3951114579 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe458158750 con 0x7fe4580cd740 2024-03-23T07:58:27.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:27.619+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe458158750 con 0x7fe4580cd740 2024-03-23T07:58:28.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.169+0000 7fe454c00640 1 -- 172.21.15.105:0/3951114579 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe458158750 con 0x7fe4580cd740 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe43806ac00 msgr2=0x7fe43806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe43806ac00 0x7fe43806d0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe45000d430 tx=0x7fe450007530 comp rx=0 tx=0).stop 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580cd740 msgr2=0x7fe4580caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580cd740 0x7fe4580caa90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c430 tx=0x7fe44400c8f0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 shutdown_connections 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4580d0d40 0x7fe4580b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4580cd740 0x7fe4580caa90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe43806ac00 0x7fe43806d0c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 --2- 172.21.15.105:0/3951114579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4580b5aa0 0x7fe4581f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 >> 172.21.15.105:0/3951114579 conn(0x7fe4580cf5d0 msgr2=0x7fe4580b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:28.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 shutdown_connections 2024-03-23T07:58:28.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.173+0000 7fe45f400640 1 -- 172.21.15.105:0/3951114579 wait complete. 2024-03-23T07:58:28.195 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T07:58:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.447+0000 7fef05e00640 1 Processor -- start 2024-03-23T07:58:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.447+0000 7fef05e00640 1 -- start start 2024-03-23T07:58:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.449+0000 7fef05e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fef00159830 0x7fef000cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.449+0000 7fef05e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef000ca7b0 0x7fef000cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.449+0000 7fef05e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef00159c10 con 0x7fef000ce930 2024-03-23T07:58:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.449+0000 7fef05e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0006de50 con 0x7fef00159830 2024-03-23T07:58:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.449+0000 7fef05e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000cd840 con 0x7fef000ca7b0 2024-03-23T07:58:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.449+0000 7feeff400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fef000ce930 0x7fef000b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43284/0 (socket says 172.21.15.105:43284) 2024-03-23T07:58:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.449+0000 7feeff400640 1 -- 172.21.15.105:0/582129906 learned_addr learned my addr 172.21.15.105:0/582129906 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:28.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefea00640 1 --2- 172.21.15.105:0/582129906 >> v2:172.21.15.132:3300/0 conn(0x7fef00159830 0x7fef000cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:28.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feef6000640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef000ca7b0 0x7fef000cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557093120 0 0) 0x7fef00159c10 con 0x7fef000ce930 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee4003520 con 0x7fef000ce930 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 616939520 0 0) 0x7feee4003520 con 0x7fef000ce930 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef000ca7b0 msgr2=0x7fef000cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef000ca7b0 0x7fef000cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 >> v2:172.21.15.132:3300/0 conn(0x7fef00159830 msgr2=0x7fef000cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 --2- 172.21.15.105:0/582129906 >> v2:172.21.15.132:3300/0 conn(0x7fef00159830 0x7fef000cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000ccc30 con 0x7fef000ce930 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7feef80019f0 con 0x7fef000ce930 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 >> v1:172.21.15.105:6789/0 conn(0x7fef000ce930 legacy=0x7fef000b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7feee4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef000ca7b0 0x7feee40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee4004d00 0x7feee4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef000cd840 con 0x7feee4003970 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee400b980 con 0x7feee4004d00 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.450+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee400baf0 con 0x7fef000ca7b0 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.451+0000 7feefea00640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7feee4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.451+0000 7feefea00640 1 -- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef000ca7b0 msgr2=0x7feee40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.451+0000 7feefea00640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef000ca7b0 0x7feee40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.451+0000 7feefea00640 1 -- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee4004d00 msgr2=0x7feee4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.451+0000 7feefea00640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee4004d00 0x7feee4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.451+0000 7feefea00640 1 -- 172.21.15.105:0/582129906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4003520 con 0x7feee4003970 2024-03-23T07:58:28.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.452+0000 7feefea00640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7feee4003d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7feef0009650 tx=0x7feee40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8527f4eff010418 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:28.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.452+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef002e920 con 0x7feee4003970 2024-03-23T07:58:28.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef002ea80 con 0x7feee4003970 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7feefe000640 1 -- 172.21.15.105:0/582129906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef002ee10 con 0x7feee4003970 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7fef05e00640 1 -- 172.21.15.105:0/582129906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 msgr2=0x7feee4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7fef05e00640 1 --2- 172.21.15.105:0/582129906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7feee4003d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7feef0009650 tx=0x7feee40057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7feeff400640 1 -- 172.21.15.105:0/582129906 reap_dead start 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7fef05e00640 1 -- 172.21.15.105:0/582129906 shutdown_connections 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7fef05e00640 1 -- 172.21.15.105:0/582129906 >> 172.21.15.105:0/582129906 conn(0x7fef0006a0b0 msgr2=0x7fef0006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7fef05e00640 1 -- 172.21.15.105:0/582129906 shutdown_connections 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.453+0000 7fef05e00640 1 -- 172.21.15.105:0/582129906 wait complete. 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.455+0000 7fef05e00640 1 Processor -- start 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.455+0000 7fef05e00640 1 -- start start 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7fef05e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7fef0013f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7fef05e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef0014f490 0x7fef00157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7fef05e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef001e4930 0x7fef001e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7fef05e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00157840 con 0x7feee4003970 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7fef05e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001e52f0 con 0x7fef001e4930 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7fef05e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000aed20 con 0x7fef0014f490 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7feefea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7fef0013f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7feefea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7fef0013f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43620/0 (socket says 172.21.15.105:43620) 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.456+0000 7feefea00640 1 -- 172.21.15.105:0/2795719873 learned_addr learned my addr 172.21.15.105:0/2795719873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feef6000640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef0014f490 0x7fef00157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feefea00640 1 -- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef0014f490 msgr2=0x7fef00157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feeff400640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef001e4930 0x7fef001e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feefea00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef0014f490 0x7fef00157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feefea00640 1 -- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef001e4930 msgr2=0x7fef001e4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:28.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feefea00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef001e4930 0x7fef001e4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:28.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feefea00640 1 -- 172.21.15.105:0/2795719873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef001574a0 con 0x7feee4003970 2024-03-23T07:58:28.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.457+0000 7feefea00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7fef0013f1f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feef0039040 tx=0x7feef0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.458+0000 7feef7e00640 1 -- 172.21.15.105:0/2795719873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0003810 con 0x7feee4003970 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.458+0000 7feef7e00640 1 -- 172.21.15.105:0/2795719873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0003970 con 0x7feee4003970 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.458+0000 7feef7e00640 1 -- 172.21.15.105:0/2795719873 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0003d00 con 0x7feee4003970 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.458+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef000ae760 con 0x7feee4003970 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.458+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef001581a0 con 0x7feee4003970 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.459+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef00157ab0 con 0x7feee4003970 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.460+0000 7feef7e00640 1 -- 172.21.15.105:0/2795719873 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feef003d070 con 0x7feee4003970 2024-03-23T07:58:28.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.460+0000 7feef7e00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feedc06ac00 0x7feedc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:28.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.460+0000 7feef7e00640 1 -- 172.21.15.105:0/2795719873 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6317+0+0 (secure 0 0 0) 0x7feef00aae50 con 0x7feee4003970 2024-03-23T07:58:28.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.461+0000 7feef6000640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feedc06ac00 0x7feedc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:28.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.461+0000 7feef6000640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feedc06ac00 0x7feedc06d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef0014f9e0 tx=0x7feee8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:28.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.465+0000 7feef7e00640 1 -- 172.21.15.105:0/2795719873 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feef0075dc0 con 0x7feee4003970 2024-03-23T07:58:28.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:28.656+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef00158360 con 0x7feee4003970 2024-03-23T07:58:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.388+0000 7feef7e00640 1 -- 172.21.15.105:0/2795719873 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 181+0+0 (secure 0 0 0) 0x7feef0003e60 con 0x7feee4003970 2024-03-23T07:58:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feedc06ac00 msgr2=0x7feedc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feedc06ac00 0x7feedc06d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef0014f9e0 tx=0x7feee8008040 comp rx=0 tx=0).stop 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 msgr2=0x7fef0013f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7fef0013f1f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feef0039040 tx=0x7feef0004730 comp rx=0 tx=0).stop 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 shutdown_connections 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef001e4930 0x7fef001e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7feedc06ac00 0x7feedc06d0c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef0014f490 0x7fef00157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 --2- 172.21.15.105:0/2795719873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feee4003970 0x7fef0013f1f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 >> 172.21.15.105:0/2795719873 conn(0x7fef0006a0b0 msgr2=0x7fef000ad730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 shutdown_connections 2024-03-23T07:58:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.392+0000 7fef05e00640 1 -- 172.21.15.105:0/2795719873 wait complete. 2024-03-23T07:58:29.415 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T07:58:29.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.620+0000 7f8eade00640 1 Processor -- start 2024-03-23T07:58:29.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.621+0000 7f8eade00640 1 -- start start 2024-03-23T07:58:29.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.622+0000 7f8eade00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8ea80ca7b0 0x7f8ea80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.622+0000 7f8eade00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea8159c10 con 0x7f8ea80ce930 2024-03-23T07:58:29.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.622+0000 7f8eade00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea80cd840 con 0x7f8ea8159830 2024-03-23T07:58:29.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.622+0000 7f8eade00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea806de50 con 0x7f8ea80ca7b0 2024-03-23T07:58:29.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea7400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8ea80ce930 0x7f8ea80b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43296/0 (socket says 172.21.15.105:43296) 2024-03-23T07:58:29.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea7400640 1 -- 172.21.15.105:0/90819749 learned_addr learned my addr 172.21.15.105:0/90819749 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:29.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637194976 0 0) 0x7f8ea8159c10 con 0x7f8ea80ce930 2024-03-23T07:58:29.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8e9e000640 1 --2- 172.21.15.105:0/90819749 >> v2:172.21.15.132:3300/0 conn(0x7f8ea80ca7b0 0x7f8ea80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e8c005730 con 0x7f8ea80ce930 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3553411741 0 0) 0x7f8e8c005730 con 0x7f8ea80ce930 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 >> v2:172.21.15.132:3300/0 conn(0x7f8ea80ca7b0 msgr2=0x7f8ea80cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea6000640 1 --2- 172.21.15.105:0/90819749 >> v2:172.21.15.132:3300/0 conn(0x7f8ea80ca7b0 0x7f8ea80cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 >> v1:172.21.15.132:6789/0 conn(0x7f8ea8159830 legacy=0x7f8ea80cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.623+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea80ccc30 con 0x7f8ea80ce930 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f8ea0001670 con 0x7f8ea80ce930 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 >> v1:172.21.15.105:6789/0 conn(0x7f8ea80ce930 legacy=0x7f8ea80b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e8c005bd0 0x7f8e8c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c0069d0 0x7f8e8c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e8c0073e0 0x7f8e8c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c00e2f0 con 0x7f8e8c005bd0 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c00e430 con 0x7f8e8c0069d0 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c00e5f0 con 0x7f8e8c0073e0 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6a00640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e8c005bd0 0x7f8e8c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8e9e000640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c0069d0 0x7f8e8c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6a00640 1 -- 172.21.15.105:0/90819749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e8c0073e0 msgr2=0x7f8e8c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6a00640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e8c0073e0 0x7f8e8c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6a00640 1 -- 172.21.15.105:0/90819749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c0069d0 msgr2=0x7f8e8c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6a00640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c0069d0 0x7f8e8c006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea7400640 1 -- 172.21.15.105:0/90819749 reap_dead start 2024-03-23T07:58:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.624+0000 7f8ea6a00640 1 -- 172.21.15.105:0/90819749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c005730 con 0x7f8e8c005bd0 2024-03-23T07:58:29.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.626+0000 7f8ea6a00640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e8c005bd0 0x7f8e8c005fd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e980044b0 tx=0x7f8e8c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511e7c690c7b0b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:29.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.626+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98030620 con 0x7f8e8c005bd0 2024-03-23T07:58:29.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.626+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98030780 con 0x7f8e8c005bd0 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.626+0000 7f8ea6000640 1 -- 172.21.15.105:0/90819749 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98030b70 con 0x7f8e8c005bd0 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.627+0000 7f8eade00640 1 -- 172.21.15.105:0/90819749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e8c005bd0 msgr2=0x7f8e8c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.627+0000 7f8eade00640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e8c005bd0 0x7f8e8c005fd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e980044b0 tx=0x7f8e8c007a00 comp rx=0 tx=0).stop 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.627+0000 7f8eade00640 1 -- 172.21.15.105:0/90819749 shutdown_connections 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.627+0000 7f8eade00640 1 --2- 172.21.15.105:0/90819749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e8c005bd0 0x7f8e8c005fd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.627+0000 7f8eade00640 1 -- 172.21.15.105:0/90819749 >> 172.21.15.105:0/90819749 conn(0x7f8ea806a0b0 msgr2=0x7f8ea806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.627+0000 7f8eade00640 1 -- 172.21.15.105:0/90819749 shutdown_connections 2024-03-23T07:58:29.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.627+0000 7f8eade00640 1 -- 172.21.15.105:0/90819749 wait complete. 2024-03-23T07:58:29.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.629+0000 7f8eade00640 1 Processor -- start 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.629+0000 7f8eade00640 1 -- start start 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8eade00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c005bd0 0x7f8ea814a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8eade00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ea80b1390 0x7f8ea80b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8eade00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea8152d60 0x7f8ea81531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8eade00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea80b1d50 con 0x7f8ea8152d60 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8eade00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8159db0 con 0x7f8e8c005bd0 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8eade00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea815a090 con 0x7f8ea80b1390 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea8152d60 0x7f8ea81531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea8152d60 0x7f8ea81531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43638/0 (socket says 172.21.15.105:43638) 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 -- 172.21.15.105:0/2672391748 learned_addr learned my addr 172.21.15.105:0/2672391748 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea6a00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c005bd0 0x7f8ea814a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:29.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8e9e000640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ea80b1390 0x7f8ea80b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 -- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ea80b1390 msgr2=0x7f8ea80b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ea80b1390 0x7f8ea80b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 -- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c005bd0 msgr2=0x7f8ea814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c005bd0 0x7f8ea814a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.630+0000 7f8ea7400640 1 -- 172.21.15.105:0/2672391748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea815a370 con 0x7f8ea8152d60 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.631+0000 7f8ea7400640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea8152d60 0x7f8ea81531e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8071360 tx=0x7f8ea000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.631+0000 7f8e9fe00640 1 -- 172.21.15.105:0/2672391748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0007230 con 0x7f8ea8152d60 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.631+0000 7f8e9fe00640 1 -- 172.21.15.105:0/2672391748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea000ced0 con 0x7f8ea8152d60 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.631+0000 7f8e9fe00640 1 -- 172.21.15.105:0/2672391748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00075d0 con 0x7f8ea8152d60 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.631+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea815a650 con 0x7f8ea8152d60 2024-03-23T07:58:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.631+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea8153720 con 0x7f8ea8152d60 2024-03-23T07:58:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.632+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea8073390 con 0x7f8ea8152d60 2024-03-23T07:58:29.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.633+0000 7f8e9fe00640 1 -- 172.21.15.105:0/2672391748 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8ea00047e0 con 0x7f8ea8152d60 2024-03-23T07:58:29.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.634+0000 7f8e9fe00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e8406ac00 0x7f8e8406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:29.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.634+0000 7f8e9fe00640 1 -- 172.21.15.105:0/2672391748 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5879+0+0 (secure 0 0 0) 0x7f8ea0088d20 con 0x7f8ea8152d60 2024-03-23T07:58:29.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.634+0000 7f8ea6a00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e8406ac00 0x7f8e8406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:29.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.635+0000 7f8ea6a00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e8406ac00 0x7f8e8406d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e98036000 tx=0x7f8e98009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:29.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.639+0000 7f8e9fe00640 1 -- 172.21.15.105:0/2672391748 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8ea0053db0 con 0x7f8ea8152d60 2024-03-23T07:58:29.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:29.829+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ea80cdde0 con 0x7f8ea8152d60 2024-03-23T07:58:30.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.402+0000 7f8e9fe00640 1 -- 172.21.15.105:0/2672391748 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ea00573d0 con 0x7f8ea8152d60 2024-03-23T07:58:30.403 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-03-23T07:58:30.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e8406ac00 msgr2=0x7f8e8406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e8406ac00 0x7f8e8406d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e98036000 tx=0x7f8e98009030 comp rx=0 tx=0).stop 2024-03-23T07:58:30.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea8152d60 msgr2=0x7f8ea81531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea8152d60 0x7f8ea81531e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8071360 tx=0x7f8ea000cb90 comp rx=0 tx=0).stop 2024-03-23T07:58:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 shutdown_connections 2024-03-23T07:58:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea8152d60 0x7f8ea81531e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f8e8406ac00 0x7f8e8406d0c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ea80b1390 0x7f8ea80b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 --2- 172.21.15.105:0/2672391748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e8c005bd0 0x7f8ea814a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.407+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 >> 172.21.15.105:0/2672391748 conn(0x7f8ea806a0b0 msgr2=0x7f8ea81542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:30.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.408+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 shutdown_connections 2024-03-23T07:58:30.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.408+0000 7f8eade00640 1 -- 172.21.15.105:0/2672391748 wait complete. 2024-03-23T07:58:30.431 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T07:58:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.627+0000 7fdf2f200640 1 Processor -- start 2024-03-23T07:58:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.627+0000 7fdf2f200640 1 -- start start 2024-03-23T07:58:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.628+0000 7fdf2f200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf28159830 0x7fdf280cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.628+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ca7b0 0x7fdf280cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.628+0000 7fdf2f200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf28159c10 con 0x7fdf280ce930 2024-03-23T07:58:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.628+0000 7fdf2f200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2806de50 con 0x7fdf28159830 2024-03-23T07:58:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.628+0000 7fdf2f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280cd840 con 0x7fdf280ca7b0 2024-03-23T07:58:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf27e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf28159830 0x7fdf280cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:30.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf27e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf28159830 0x7fdf280cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43646/0 (socket says 172.21.15.105:43646) 2024-03-23T07:58:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf27e00640 1 -- 172.21.15.105:0/1613596491 learned_addr learned my addr 172.21.15.105:0/1613596491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf2d200640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdf280ce930 0x7fdf280b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43312/0 (socket says 172.21.15.105:43312) 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf1f400640 1 --2- 172.21.15.105:0/1613596491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ca7b0 0x7fdf280cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf27e00640 1 -- 172.21.15.105:0/1613596491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ca7b0 msgr2=0x7fdf280cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf27e00640 1 --2- 172.21.15.105:0/1613596491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ca7b0 0x7fdf280cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf27e00640 1 -- 172.21.15.105:0/1613596491 >> v1:172.21.15.105:6789/0 conn(0x7fdf280ce930 legacy=0x7fdf280b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.629+0000 7fdf27e00640 1 -- 172.21.15.105:0/1613596491 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf280ccc30 con 0x7fdf28159830 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.630+0000 7fdf27e00640 1 --2- 172.21.15.105:0/1613596491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf28159830 0x7fdf280cc3c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85af65f36631ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.630+0000 7fdf27400640 1 -- 172.21.15.105:0/1613596491 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1802f830 con 0x7fdf28159830 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf27400640 1 -- 172.21.15.105:0/1613596491 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1802f990 con 0x7fdf28159830 2024-03-23T07:58:30.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf27400640 1 -- 172.21.15.105:0/1613596491 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1802fd20 con 0x7fdf28159830 2024-03-23T07:58:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 -- 172.21.15.105:0/1613596491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf28159830 msgr2=0x7fdf280cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 --2- 172.21.15.105:0/1613596491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf28159830 0x7fdf280cc3c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802ba10 comp rx=0 tx=0).stop 2024-03-23T07:58:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 -- 172.21.15.105:0/1613596491 shutdown_connections 2024-03-23T07:58:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 --2- 172.21.15.105:0/1613596491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ca7b0 0x7fdf280cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 --2- 172.21.15.105:0/1613596491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf28159830 0x7fdf280cc3c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 -- 172.21.15.105:0/1613596491 >> 172.21.15.105:0/1613596491 conn(0x7fdf2806a0b0 msgr2=0x7fdf2806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 -- 172.21.15.105:0/1613596491 shutdown_connections 2024-03-23T07:58:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.631+0000 7fdf2f200640 1 -- 172.21.15.105:0/1613596491 wait complete. 2024-03-23T07:58:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.633+0000 7fdf2f200640 1 Processor -- start 2024-03-23T07:58:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.633+0000 7fdf2f200640 1 -- start start 2024-03-23T07:58:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf280ca7b0 0x7fdf28151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ce930 0x7fdf280aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf28159830 0x7fdf281587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf2f200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28151b70 con 0x7fdf280ca7b0 2024-03-23T07:58:30.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf2f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280af0f0 con 0x7fdf28159830 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf2f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280ae720 con 0x7fdf280ce930 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf280ca7b0 0x7fdf28151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf280ca7b0 0x7fdf28151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43656/0 (socket says 172.21.15.105:43656) 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.634+0000 7fdf27e00640 1 -- 172.21.15.105:0/750113585 learned_addr learned my addr 172.21.15.105:0/750113585 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf27e00640 1 -- 172.21.15.105:0/750113585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ce930 msgr2=0x7fdf280aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf27e00640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ce930 0x7fdf280aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf27e00640 1 -- 172.21.15.105:0/750113585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf28159830 msgr2=0x7fdf281587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf27e00640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf28159830 0x7fdf281587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf27e00640 1 -- 172.21.15.105:0/750113585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180094d0 con 0x7fdf280ca7b0 2024-03-23T07:58:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf27e00640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf280ca7b0 0x7fdf28151630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdf1802ba50 tx=0x7fdf18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf24c00640 1 -- 172.21.15.105:0/750113585 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18037c40 con 0x7fdf280ca7b0 2024-03-23T07:58:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf24c00640 1 -- 172.21.15.105:0/750113585 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18037da0 con 0x7fdf280ca7b0 2024-03-23T07:58:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf280adb10 con 0x7fdf280ca7b0 2024-03-23T07:58:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf24c00640 1 -- 172.21.15.105:0/750113585 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf180426c0 con 0x7fdf280ca7b0 2024-03-23T07:58:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.635+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf280ae0d0 con 0x7fdf280ca7b0 2024-03-23T07:58:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.636+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf28073390 con 0x7fdf280ca7b0 2024-03-23T07:58:30.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.637+0000 7fdf24c00640 1 -- 172.21.15.105:0/750113585 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdf1802d3c0 con 0x7fdf280ca7b0 2024-03-23T07:58:30.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.638+0000 7fdf24c00640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf0406ac30 0x7fdf0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.638+0000 7fdf24c00640 1 -- 172.21.15.105:0/750113585 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fdf1803c070 con 0x7fdf280ca7b0 2024-03-23T07:58:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.638+0000 7fdf1f400640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf0406ac30 0x7fdf0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.639+0000 7fdf1f400640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf0406ac30 0x7fdf0406d0f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdf280ac9b0 tx=0x7fdf10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:30.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.642+0000 7fdf24c00640 1 -- 172.21.15.105:0/750113585 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdf18076c90 con 0x7fdf280ca7b0 2024-03-23T07:58:30.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.844+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdf2806a7d0 con 0x7fdf280ca7b0 2024-03-23T07:58:30.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.850+0000 7fdf24c00640 1 -- 172.21.15.105:0/750113585 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf1807a2b0 con 0x7fdf280ca7b0 2024-03-23T07:58:30.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.853+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf0406ac30 msgr2=0x7fdf0406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.853+0000 7fdf2f200640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf0406ac30 0x7fdf0406d0f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdf280ac9b0 tx=0x7fdf10008040 comp rx=0 tx=0).stop 2024-03-23T07:58:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.853+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf280ca7b0 msgr2=0x7fdf28151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.853+0000 7fdf2f200640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf280ca7b0 0x7fdf28151630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdf1802ba50 tx=0x7fdf18004730 comp rx=0 tx=0).stop 2024-03-23T07:58:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 shutdown_connections 2024-03-23T07:58:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf28159830 0x7fdf281587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fdf0406ac30 0x7fdf0406d0f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf280ce930 0x7fdf280aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 --2- 172.21.15.105:0/750113585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf280ca7b0 0x7fdf28151630 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:30.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 >> 172.21.15.105:0/750113585 conn(0x7fdf2806a0b0 msgr2=0x7fdf281563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:30.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 shutdown_connections 2024-03-23T07:58:30.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:30.854+0000 7fdf2f200640 1 -- 172.21.15.105:0/750113585 wait complete. 2024-03-23T07:58:30.874 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-03-23T07:58:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.084+0000 7fe91f200640 1 Processor -- start 2024-03-23T07:58:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.084+0000 7fe91f200640 1 -- start start 2024-03-23T07:58:31.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.085+0000 7fe91f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe91815cda0 0x7fe9180cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:31.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.085+0000 7fe91f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9180b5a30 0x7fe9180b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:31.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.086+0000 7fe91f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9180d1830 con 0x7fe9180b4350 2024-03-23T07:58:31.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.086+0000 7fe91f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9180b1f20 con 0x7fe91815cda0 2024-03-23T07:58:31.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.086+0000 7fe91f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180d0c20 con 0x7fe9180b5a30 2024-03-23T07:58:31.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.086+0000 7fe90f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9180b5a30 0x7fe9180b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:31.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.086+0000 7fe90f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9180b5a30 0x7fe9180b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:40656/0 (socket says 172.21.15.105:40656) 2024-03-23T07:58:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.086+0000 7fe90f400640 1 -- 172.21.15.105:0/1101634032 learned_addr learned my addr 172.21.15.105:0/1101634032 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.086+0000 7fe917e00640 1 --2- 172.21.15.105:0/1101634032 >> v2:172.21.15.132:3300/0 conn(0x7fe91815cda0 0x7fe9180cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.087+0000 7fe90f400640 1 -- 172.21.15.105:0/1101634032 >> v2:172.21.15.132:3300/0 conn(0x7fe91815cda0 msgr2=0x7fe9180cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.087+0000 7fe90f400640 1 --2- 172.21.15.105:0/1101634032 >> v2:172.21.15.132:3300/0 conn(0x7fe91815cda0 0x7fe9180cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.087+0000 7fe90f400640 1 -- 172.21.15.105:0/1101634032 >> v1:172.21.15.132:6789/0 conn(0x7fe9180b4350 legacy=0x7fe9180d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.087+0000 7fe90f400640 1 -- 172.21.15.105:0/1101634032 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180d0f00 con 0x7fe9180b5a30 2024-03-23T07:58:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.088+0000 7fe90f400640 1 --2- 172.21.15.105:0/1101634032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9180b5a30 0x7fe9180b5e90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe900009870 tx=0x7fe90002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923da436d0dbfe73 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:31.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe917400640 1 -- 172.21.15.105:0/1101634032 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900030840 con 0x7fe9180b5a30 2024-03-23T07:58:31.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe917400640 1 -- 172.21.15.105:0/1101634032 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9000309a0 con 0x7fe9180b5a30 2024-03-23T07:58:31.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe917400640 1 -- 172.21.15.105:0/1101634032 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900030d30 con 0x7fe9180b5a30 2024-03-23T07:58:31.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 -- 172.21.15.105:0/1101634032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9180b5a30 msgr2=0x7fe9180b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:31.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 --2- 172.21.15.105:0/1101634032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9180b5a30 0x7fe9180b5e90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe900009870 tx=0x7fe90002b980 comp rx=0 tx=0).stop 2024-03-23T07:58:31.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 -- 172.21.15.105:0/1101634032 shutdown_connections 2024-03-23T07:58:31.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 --2- 172.21.15.105:0/1101634032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9180b5a30 0x7fe9180b5e90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 --2- 172.21.15.105:0/1101634032 >> v2:172.21.15.132:3300/0 conn(0x7fe91815cda0 0x7fe9180cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 -- 172.21.15.105:0/1101634032 >> 172.21.15.105:0/1101634032 conn(0x7fe9180cf350 msgr2=0x7fe9180cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:31.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 -- 172.21.15.105:0/1101634032 shutdown_connections 2024-03-23T07:58:31.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.089+0000 7fe91f200640 1 -- 172.21.15.105:0/1101634032 wait complete. 2024-03-23T07:58:31.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.091+0000 7fe91f200640 1 Processor -- start 2024-03-23T07:58:31.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.091+0000 7fe91f200640 1 -- start start 2024-03-23T07:58:31.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe91f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180b4350 0x7fe91814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:31.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe91f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9180b5a30 0x7fe9180c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:31.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe91f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91815cda0 0x7fe9180d2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:31.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe91f200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91814d2c0 con 0x7fe9180b5a30 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe91f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180c75a0 con 0x7fe9180b4350 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe91f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180c6ec0 con 0x7fe91815cda0 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe90f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9180b5a30 0x7fe9180c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe90f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9180b5a30 0x7fe9180c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43658/0 (socket says 172.21.15.105:43658) 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.092+0000 7fe90f400640 1 -- 172.21.15.105:0/19400044 learned_addr learned my addr 172.21.15.105:0/19400044 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe917e00640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180b4350 0x7fe91814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe91d200640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91815cda0 0x7fe9180d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe90f400640 1 -- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91815cda0 msgr2=0x7fe9180d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe90f400640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91815cda0 0x7fe9180d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe90f400640 1 -- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180b4350 msgr2=0x7fe91814cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe90f400640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180b4350 0x7fe91814cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe90f400640 1 -- 172.21.15.105:0/19400044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000094d0 con 0x7fe9180b5a30 2024-03-23T07:58:31.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.093+0000 7fe90f400640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9180b5a30 0x7fe9180c6400 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe900009870 tx=0x7fe900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:31.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.094+0000 7fe914c00640 1 -- 172.21.15.105:0/19400044 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900039c90 con 0x7fe9180b5a30 2024-03-23T07:58:31.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.094+0000 7fe914c00640 1 -- 172.21.15.105:0/19400044 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe900039e10 con 0x7fe9180b5a30 2024-03-23T07:58:31.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.094+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9180c71a0 con 0x7fe9180b5a30 2024-03-23T07:58:31.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.094+0000 7fe914c00640 1 -- 172.21.15.105:0/19400044 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900043640 con 0x7fe9180b5a30 2024-03-23T07:58:31.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.094+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9180c7ad0 con 0x7fe9180b5a30 2024-03-23T07:58:31.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.095+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe918073390 con 0x7fe9180b5a30 2024-03-23T07:58:31.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.096+0000 7fe914c00640 1 -- 172.21.15.105:0/19400044 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe9000437a0 con 0x7fe9180b5a30 2024-03-23T07:58:31.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.096+0000 7fe914c00640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe8f406ac00 0x7fe8f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:31.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.097+0000 7fe914c00640 1 -- 172.21.15.105:0/19400044 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fe90003e070 con 0x7fe9180b5a30 2024-03-23T07:58:31.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.097+0000 7fe917e00640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe8f406ac00 0x7fe8f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:31.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.097+0000 7fe917e00640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe8f406ac00 0x7fe8f406d0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe908009a30 tx=0x7fe908008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:31.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.101+0000 7fe914c00640 1 -- 172.21.15.105:0/19400044 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe900077520 con 0x7fe9180b5a30 2024-03-23T07:58:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.364+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) v1 -- 0x7fe9180c7390 con 0x7fe9180b5a30 2024-03-23T07:58:31.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.402+0000 7fe914c00640 1 -- 172.21.15.105:0/19400044 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v138) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe90007ab40 con 0x7fe9180b5a30 2024-03-23T07:58:31.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.404+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe8f406ac00 msgr2=0x7fe8f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.404+0000 7fe91f200640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe8f406ac00 0x7fe8f406d0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe908009a30 tx=0x7fe908008040 comp rx=0 tx=0).stop 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.404+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9180b5a30 msgr2=0x7fe9180c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9180b5a30 0x7fe9180c6400 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe900009870 tx=0x7fe900004730 comp rx=0 tx=0).stop 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 shutdown_connections 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fe8f406ac00 0x7fe8f406d0c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91815cda0 0x7fe9180d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9180b5a30 0x7fe9180c6400 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 --2- 172.21.15.105:0/19400044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180b4350 0x7fe91814cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 >> 172.21.15.105:0/19400044 conn(0x7fe9180cf350 msgr2=0x7fe9180b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 shutdown_connections 2024-03-23T07:58:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:31.405+0000 7fe91f200640 1 -- 172.21.15.105:0/19400044 wait complete. 2024-03-23T07:58:31.423 INFO:tasks.cephfs_test_runner:MDS starts to handle client caps when it enters clientreplay stage. ... ok 2024-03-23T07:58:31.423 INFO:tasks.cephfs_test_runner: 2024-03-23T07:58:31.423 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-23T07:58:31.424 INFO:tasks.cephfs_test_runner:Ran 1 test in 69.908s 2024-03-23T07:58:31.424 INFO:tasks.cephfs_test_runner: 2024-03-23T07:58:31.424 INFO:tasks.cephfs_test_runner:OK 2024-03-23T07:58:31.424 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-23T07:58:31.432 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-23T07:58:31.440 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-23T07:58:31.440 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:58:31.440 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:31.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053428,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T07:58:31.493 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-23T07:58:31.493 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:31.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053428,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-23T07:58:31.543 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:58:31.543 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:31.601 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T07:58:31.602 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-23T07:58:31.602 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T07:58:31.652 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-23T07:58:31.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:31.652 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:58:31.708 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T07:58:31.708 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-23T07:58:31.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:31.709 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T07:58:31.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:31.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-23T07:58:31.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:31.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-23T07:58:32.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.029 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-23T07:58:32.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-23T07:58:32.189 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-03-23T07:58:32.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:58:32.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.193 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-23T07:58:32.249 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-23T07:58:32.249 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-23T07:58:32.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.250 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-23T07:58:32.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-23T07:58:32.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-03-23T07:58:32.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.554 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-23T07:58:32.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T07:58:32.599 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-03-23T07:58:32.731 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-03-23T07:58:32.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:58:32.736 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-23T07:58:32.745 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-23T07:58:32.746 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:58:32.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.961+0000 7f2937400640 1 Processor -- start 2024-03-23T07:58:32.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.962+0000 7f2937400640 1 -- start start 2024-03-23T07:58:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.963+0000 7f2937400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29300b39b0 0x7f29300b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.963+0000 7f2937400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293015c400 0x7f293015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.963+0000 7f2937400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293006c6f0 con 0x7f293015b560 2024-03-23T07:58:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.963+0000 7f2937400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29300b4370 con 0x7f29300b39b0 2024-03-23T07:58:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.963+0000 7f2937400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293006c9d0 con 0x7f293015c400 2024-03-23T07:58:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.964+0000 7f2935400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293015c400 0x7f293015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.964+0000 7f2935400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293015c400 0x7f293015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:40666/0 (socket says 172.21.15.105:40666) 2024-03-23T07:58:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.964+0000 7f2935400640 1 -- 172.21.15.105:0/468583298 learned_addr learned my addr 172.21.15.105:0/468583298 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.964+0000 7f2927e00640 1 --2- 172.21.15.105:0/468583298 >> v2:172.21.15.132:3300/0 conn(0x7f29300b39b0 0x7f29300b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.965+0000 7f2935400640 1 -- 172.21.15.105:0/468583298 >> v2:172.21.15.132:3300/0 conn(0x7f29300b39b0 msgr2=0x7f29300b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.965+0000 7f2935400640 1 --2- 172.21.15.105:0/468583298 >> v2:172.21.15.132:3300/0 conn(0x7f29300b39b0 0x7f29300b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.965+0000 7f2935400640 1 -- 172.21.15.105:0/468583298 >> v1:172.21.15.132:6789/0 conn(0x7f293015b560 legacy=0x7f293015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.965+0000 7f2935400640 1 -- 172.21.15.105:0/468583298 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293006ccb0 con 0x7f293015c400 2024-03-23T07:58:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.966+0000 7f2935400640 1 --2- 172.21.15.105:0/468583298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293015c400 0x7f293015c870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2928009e50 tx=0x7f292802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71aee5cb5c274c2e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.967+0000 7f292fe00640 1 -- 172.21.15.105:0/468583298 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292802cb00 con 0x7f293015c400 2024-03-23T07:58:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.967+0000 7f292fe00640 1 -- 172.21.15.105:0/468583298 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292802cc60 con 0x7f293015c400 2024-03-23T07:58:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.967+0000 7f292fe00640 1 -- 172.21.15.105:0/468583298 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29280343f0 con 0x7f293015c400 2024-03-23T07:58:32.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.967+0000 7f2937400640 1 -- 172.21.15.105:0/468583298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293015c400 msgr2=0x7f293015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:32.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.967+0000 7f2937400640 1 --2- 172.21.15.105:0/468583298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293015c400 0x7f293015c870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2928009e50 tx=0x7f292802bf60 comp rx=0 tx=0).stop 2024-03-23T07:58:32.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.968+0000 7f2937400640 1 -- 172.21.15.105:0/468583298 shutdown_connections 2024-03-23T07:58:32.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.968+0000 7f2937400640 1 --2- 172.21.15.105:0/468583298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293015c400 0x7f293015c870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:32.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.968+0000 7f2937400640 1 --2- 172.21.15.105:0/468583298 >> v2:172.21.15.132:3300/0 conn(0x7f29300b39b0 0x7f29300b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:32.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.968+0000 7f2937400640 1 -- 172.21.15.105:0/468583298 >> 172.21.15.105:0/468583298 conn(0x7f293006a3d0 msgr2=0x7f29300b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:32.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.968+0000 7f2937400640 1 -- 172.21.15.105:0/468583298 shutdown_connections 2024-03-23T07:58:32.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.968+0000 7f2937400640 1 -- 172.21.15.105:0/468583298 wait complete. 2024-03-23T07:58:32.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.970+0000 7f2937400640 1 Processor -- start 2024-03-23T07:58:32.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.970+0000 7f2937400640 1 -- start start 2024-03-23T07:58:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2937400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29300b39b0 0x7f29300cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2937400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f293015b560 0x7f2930159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2937400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293015c400 0x7f293015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2937400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29301e4320 con 0x7f293015b560 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2937400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29301e4530 con 0x7f293015c400 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2937400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29301e4810 con 0x7f29300b39b0 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2927e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f293015b560 0x7f2930159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2927e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f293015b560 0x7f2930159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43674/0 (socket says 172.21.15.105:43674) 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.971+0000 7f2927e00640 1 -- 172.21.15.105:0/80814280 learned_addr learned my addr 172.21.15.105:0/80814280 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2934a00640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29300b39b0 0x7f29300cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2935400640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293015c400 0x7f293015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2927e00640 1 -- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29300b39b0 msgr2=0x7f29300cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2927e00640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29300b39b0 0x7f29300cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2927e00640 1 -- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293015c400 msgr2=0x7f293015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:32.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2927e00640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293015c400 0x7f293015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:32.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2927e00640 1 -- 172.21.15.105:0/80814280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2928009b30 con 0x7f293015b560 2024-03-23T07:58:32.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f2927e00640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f293015b560 0x7f2930159af0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f292000e870 tx=0x7f292000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:32.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.972+0000 7f292d600640 1 -- 172.21.15.105:0/80814280 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29200094f0 con 0x7f293015b560 2024-03-23T07:58:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.973+0000 7f292d600640 1 -- 172.21.15.105:0/80814280 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292000c940 con 0x7f293015b560 2024-03-23T07:58:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.973+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29301e4af0 con 0x7f293015b560 2024-03-23T07:58:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.973+0000 7f292d600640 1 -- 172.21.15.105:0/80814280 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000ccf0 con 0x7f293015b560 2024-03-23T07:58:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.973+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29300af4f0 con 0x7f293015b560 2024-03-23T07:58:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.974+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293006ac20 con 0x7f293015b560 2024-03-23T07:58:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.975+0000 7f292d600640 1 -- 172.21.15.105:0/80814280 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2920013470 con 0x7f293015b560 2024-03-23T07:58:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.976+0000 7f292d600640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f290c06ac00 0x7f290c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.976+0000 7f292d600640 1 -- 172.21.15.105:0/80814280 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f2920089410 con 0x7f293015b560 2024-03-23T07:58:32.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.976+0000 7f2934a00640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f290c06ac00 0x7f290c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:32.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.976+0000 7f2934a00640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f290c06ac00 0x7f290c06d0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2918000f60 tx=0x7f2918009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:32.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:32.980+0000 7f292d600640 1 -- 172.21.15.105:0/80814280 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2920054620 con 0x7f293015b560 2024-03-23T07:58:33.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.172+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f293006dd80 con 0x7f290c06ac00 2024-03-23T07:58:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.178+0000 7f292d600640 1 -- 172.21.15.105:0/80814280 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48413 (secure 0 0 0) 0x7f29180050d0 con 0x7f290c06ac00 2024-03-23T07:58:33.179 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:58:33.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f290c06ac00 msgr2=0x7f290c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f290c06ac00 0x7f290c06d0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2918000f60 tx=0x7f2918009040 comp rx=0 tx=0).stop 2024-03-23T07:58:33.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f293015b560 msgr2=0x7f2930159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f293015b560 0x7f2930159af0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f292000e870 tx=0x7f292000ed30 comp rx=0 tx=0).stop 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 shutdown_connections 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293015c400 0x7f293015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f293015b560 0x7f2930159af0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f290c06ac00 0x7f290c06d0c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 --2- 172.21.15.105:0/80814280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29300b39b0 0x7f29300cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 >> 172.21.15.105:0/80814280 conn(0x7f293006a3d0 msgr2=0x7f293015b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 shutdown_connections 2024-03-23T07:58:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.181+0000 7f2937400640 1 -- 172.21.15.105:0/80814280 wait complete. 2024-03-23T07:58:33.199 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-03-23T07:58:32.086781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":221400,"kb_used_data":5244,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753320,"statfs":{"total":773094113280,"available":772867399680,"internally_reserved":0,"allocated":5369856,"data_stored":2994010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16291,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-73,"num_write_kb":-41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001797"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-23T07:58:07.112003+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:58:07.112003+0000","last_peered":"2024-03-23T07:58:07.112003+0000","last_clean":"2024-03-23T07:58:07.112003+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:58:07.112003+0000","last_undegraded":"2024-03-23T07:58:07.112003+0000","last_fullsized":"2024-03-23T07:58:07.112003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476767,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":217089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"min":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3120000000000001,"5min":1.506,"15min":1.506},"last":0.84599999999999997}]},{"osd":1,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.872,"15min":0.88},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3140000000000001,"5min":1.827,"15min":1.827},"last":0.89600000000000002},{"interface":"front","average":{"1min":1,"5min":1.0049999999999999,"15min":1.006},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.46,"5min":1.861,"15min":1.861},"last":1.4039999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.498,"5min":1.944,"15min":1.944},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.093,"15min":1.1040000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.405,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.747}]},{"osd":4,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2490000000000001,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6910000000000001,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.272},{"interface":"front","average":{"1min":1.232,"5min":1.383,"15min":1.4079999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.591,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.056}]},{"osd":5,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.466,"15min":1.498},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.6419999999999999,"5min":3.589,"15min":3.589},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.389,"15min":1.415},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.734,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.546}]},{"osd":6,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.885,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.514,"15min":1.5469999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7130000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.5089999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.389,"15min":1.399},"min":{"1min":0.78100000000000003,"5min":0.753,"15min":0.753},"max":{"1min":1.8160000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7909999999999999,"5min":2.492,"15min":2.492},"last":1.1930000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476767,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":532480,"data_stored":234944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89800000000000002,"15min":0.91400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5229999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.94399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.54700000000000004}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.077,"15min":1.117},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1930000000000001,"5min":2.04,"15min":2.04},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.129,"15min":1.159},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.661,"5min":1.839,"15min":1.839},"last":0.82299999999999995}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.056,"15min":1.085},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7689999999999999,"5min":2.081,"15min":2.081},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.288,"15min":1.331},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.585,"5min":2.012,"15min":2.012},"last":0.95999999999999996}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.4239999999999999,"15min":1.458},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8560000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.129},{"interface":"front","average":{"1min":1.206,"5min":1.3959999999999999,"15min":1.4279999999999999},"min":{"1min":0.67200000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9279999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.93100000000000005}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4610000000000001},"min":{"1min":0.89500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.4990000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.369},{"interface":"front","average":{"1min":1.339,"5min":1.571,"15min":1.609},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9470000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":1.1679999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.504,"15min":1.5449999999999999},"min":{"1min":0.85299999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0249999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.329},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.5},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9610000000000001,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.2609999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.522,"15min":1.556},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.1299999999999999,"5min":3.069,"15min":3.069},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.544,"15min":1.5960000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.0750000000000002,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.4690000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476767,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":807389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3340000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.45,"5min":3.27,"15min":3.27},"last":0.40000000000000002}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.96799999999999997,"15min":0.98099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2110000000000001,"5min":2.758,"15min":2.758},"last":0.309},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0840000000000001,"15min":1.1100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.55,"5min":3.1699999999999999,"15min":3.1699999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.163,"15min":1.1950000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.405,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.1459999999999999,"15min":1.1679999999999999},"min":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3029999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.6080000000000001,"15min":1.6739999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6870000000000001,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.347,"15min":1.377},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5589999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.73299999999999998}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5569999999999999,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.411,"15min":1.4350000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.869,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.88900000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.5580000000000001,"15min":1.6080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6870000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4790000000000001,"15min":1.5129999999999999},"min":{"1min":0.84799999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7829999999999999,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.6140000000000001,"15min":1.6639999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.75,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.875},{"interface":"front","average":{"1min":1.339,"5min":1.512,"15min":1.5409999999999999},"min":{"1min":0.85899999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.9650000000000001,"5min":3.145,"15min":3.145},"last":1.02}]}]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":507904,"data_stored":217091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1359999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.514,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4159999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.622}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.97299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3320000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.995},{"interface":"front","average":{"1min":1.052,"5min":1.153,"15min":1.1699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4970000000000001,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.86299999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.238,"15min":1.244},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7070000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.272},{"interface":"front","average":{"1min":1.149,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.8169999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1739999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.294,"15min":1.304},"min":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9590000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.3440000000000001,"15min":1.359},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9059999999999999,"5min":2.867,"15min":2.867},"last":1.089}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3939999999999999,"15min":1.4019999999999999},"min":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6759999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.3420000000000001,"15min":1.361},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6719999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4690000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.355,"15min":1.373},"min":{"1min":0.89000000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7669999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.52},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3919999999999999,"15min":1.4139999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7130000000000001,"5min":2.411,"15min":2.411},"last":1.415}]}]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":230127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.044,"15min":1.05},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.127},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.107,"15min":1.115},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.351}]},{"osd":1,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2030000000000001,"15min":1.2230000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5189999999999999,"5min":1.893,"15min":1.893},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.1559999999999999,"15min":1.1879999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.091}]},{"osd":2,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.2809999999999999,"15min":1.3120000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5600000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.2989999999999999,"15min":1.3080000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.661,"5min":1.996,"15min":1.996},"last":1.0489999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.581,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.377,"15min":1.403},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6779999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.5269999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.351},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.784,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.391},{"interface":"front","average":{"1min":1.198,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7150000000000001,"5min":2.198,"15min":2.198},"last":1.4259999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.452,"15min":1.468},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.0350000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.51,"15min":1.526},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8420000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.147}]},{"osd":6,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.591,"15min":1.6020000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.008,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.5660000000000001,"15min":1.5820000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.895,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.29}]}]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":0.99199999999999999,"15min":0.97099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0209999999999999,"15min":1.0229999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3660000000000001,"5min":1.387,"15min":1.387},"last":1.1359999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.746,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.776},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.159,"15min":1.1719999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8009999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.97199999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1819999999999999,"15min":1.1839999999999999},"min":{"1min":0.82899999999999996,"5min":0.624,"15min":0.624},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1599999999999999,"15min":1.1519999999999999},"min":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6200000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.548}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":0.97199999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":2.109},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.302,"15min":1.3089999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5589999999999999,"5min":1.968,"15min":1.968},"last":1.379}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.3009999999999999},"min":{"1min":1.0209999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6639999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.716},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.296,"15min":1.284},"min":{"1min":1.0489999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9319999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.196}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.446,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.073,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.075},{"interface":"front","average":{"1min":1.45,"5min":1.466,"15min":1.468},"min":{"1min":1.1799999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":1.804,"5min":2.194,"15min":2.194},"last":1.7789999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4910000000000001,"15min":1.5},"min":{"1min":1.1299999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.72,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.655},{"interface":"front","average":{"1min":1.508,"5min":1.546,"15min":1.552},"min":{"1min":1.2490000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.8899999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.4419999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":230127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.45},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0660000000000001,"15min":1.0669999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.141}]},{"osd":1,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.2989999999999999,"15min":1.339},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5269999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.599},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1759999999999999,"15min":1.194},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2829999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1859999999999999,"15min":1.1970000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5629999999999999,"5min":1.915,"15min":1.915},"last":1.698},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.323,"15min":1.3560000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.478,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.196}]},{"osd":3,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2370000000000001,"15min":1.254},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4379999999999999,"5min":1.734,"15min":1.734},"last":1.018},{"interface":"front","average":{"1min":1.248,"5min":1.387,"15min":1.4099999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.8240000000000001,"5min":1.988,"15min":1.988},"last":1.228}]},{"osd":5,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.347,"15min":1.365},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6379999999999999,"5min":2.198,"15min":2.198},"last":1.339},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.383,"15min":1.413},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.081,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.3759999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.581,"15min":1.613},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9650000000000001,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4630000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5980000000000001,"15min":1.6200000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.121,"5min":2.403,"15min":2.403},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.52,"15min":1.548},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8720000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.504}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":821077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0660000000000001,"15min":1.0920000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.401,"15min":1.401},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.995,"5min":1.097,"15min":1.1140000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.0860000000000001}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1830000000000001,"15min":1.206},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5009999999999999,"5min":1.663,"15min":1.663},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1950000000000001,"15min":1.212},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5580000000000001,"5min":1.833,"15min":1.833},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3120000000000001,"15min":1.3220000000000001},"min":{"1min":0.78000000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.911,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1459999999999999,"15min":1.1619999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5449999999999999,"5min":1.72,"15min":1.72},"last":1.1539999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.254,"15min":1.272},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.583},{"interface":"front","average":{"1min":1.254,"5min":1.3009999999999999,"15min":1.3089999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8220000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3220000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.274,"15min":1.254},"min":{"1min":0.90500000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.484},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.351,"15min":1.369},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.589,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.7}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.456},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.744},{"interface":"front","average":{"1min":1.284,"5min":1.3680000000000001,"15min":1.3819999999999999},"min":{"1min":0.72599999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.724,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.877}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4790000000000001,"15min":1.504},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8580000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.798},{"interface":"front","average":{"1min":1.512,"5min":1.5860000000000001,"15min":1.5980000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1200000000000001,"5min":2.282,"15min":2.282},"last":1.9259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:58:33.201 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:58:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.411+0000 7f918f000640 1 Processor -- start 2024-03-23T07:58:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.411+0000 7f918f000640 1 -- start start 2024-03-23T07:58:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.412+0000 7f918f000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f91880adbc0 0x7f91880b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.412+0000 7f918f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880cbab0 0x7f91880b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.413+0000 7f918f000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880b16a0 con 0x7f918806d390 2024-03-23T07:58:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.413+0000 7f918f000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880b4700 con 0x7f91880adbc0 2024-03-23T07:58:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.413+0000 7f918f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880cbfb0 con 0x7f91880cbab0 2024-03-23T07:58:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.413+0000 7f9187400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880cbab0 0x7f91880b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.413+0000 7f9187400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880cbab0 0x7f91880b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:40686/0 (socket says 172.21.15.105:40686) 2024-03-23T07:58:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.413+0000 7f9187400640 1 -- 172.21.15.105:0/3809732258 learned_addr learned my addr 172.21.15.105:0/3809732258 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.413+0000 7f9187e00640 1 --2- 172.21.15.105:0/3809732258 >> v2:172.21.15.132:3300/0 conn(0x7f91880adbc0 0x7f91880b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.414+0000 7f9187400640 1 -- 172.21.15.105:0/3809732258 >> v2:172.21.15.132:3300/0 conn(0x7f91880adbc0 msgr2=0x7f91880b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.414+0000 7f9187400640 1 --2- 172.21.15.105:0/3809732258 >> v2:172.21.15.132:3300/0 conn(0x7f91880adbc0 0x7f91880b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.414+0000 7f9187400640 1 -- 172.21.15.105:0/3809732258 >> v1:172.21.15.132:6789/0 conn(0x7f918806d390 legacy=0x7f91880b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.414+0000 7f9187400640 1 -- 172.21.15.105:0/3809732258 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880cc290 con 0x7f91880cbab0 2024-03-23T07:58:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.415+0000 7f9187400640 1 --2- 172.21.15.105:0/3809732258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880cbab0 0x7f91880b20f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9174009870 tx=0x7f917402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c6ad2c440e96e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:33.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.416+0000 7f9186a00640 1 -- 172.21.15.105:0/3809732258 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174031640 con 0x7f91880cbab0 2024-03-23T07:58:33.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.416+0000 7f9186a00640 1 -- 172.21.15.105:0/3809732258 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91740317a0 con 0x7f91880cbab0 2024-03-23T07:58:33.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.416+0000 7f9186a00640 1 -- 172.21.15.105:0/3809732258 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174031b30 con 0x7f91880cbab0 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.416+0000 7f918f000640 1 -- 172.21.15.105:0/3809732258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880cbab0 msgr2=0x7f91880b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.416+0000 7f918f000640 1 --2- 172.21.15.105:0/3809732258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880cbab0 0x7f91880b20f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9174009870 tx=0x7f917402b8f0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.417+0000 7f918f000640 1 -- 172.21.15.105:0/3809732258 shutdown_connections 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.417+0000 7f918f000640 1 --2- 172.21.15.105:0/3809732258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880cbab0 0x7f91880b20f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.417+0000 7f918f000640 1 --2- 172.21.15.105:0/3809732258 >> v2:172.21.15.132:3300/0 conn(0x7f91880adbc0 0x7f91880b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.417+0000 7f918f000640 1 -- 172.21.15.105:0/3809732258 >> 172.21.15.105:0/3809732258 conn(0x7f918806a3d0 msgr2=0x7f91880b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.417+0000 7f918f000640 1 -- 172.21.15.105:0/3809732258 shutdown_connections 2024-03-23T07:58:33.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.417+0000 7f918f000640 1 -- 172.21.15.105:0/3809732258 wait complete. 2024-03-23T07:58:33.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.418+0000 7f918f000640 1 Processor -- start 2024-03-23T07:58:33.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.419+0000 7f918f000640 1 -- start start 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.419+0000 7f918f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f918806d390 0x7f918813e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880adbc0 0x7f91881e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918f000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880cbab0 0x7f91881f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918f000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880ae690 con 0x7f91880cbab0 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880ca490 con 0x7f918806d390 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918813eaf0 con 0x7f91880adbc0 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880cbab0 0x7f91881f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880cbab0 0x7f91881f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43684/0 (socket says 172.21.15.105:43684) 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 -- 172.21.15.105:0/414928845 learned_addr learned my addr 172.21.15.105:0/414928845 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f9187e00640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f918806d390 0x7f918813e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f9187400640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880adbc0 0x7f91881e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 -- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880adbc0 msgr2=0x7f91881e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880adbc0 0x7f91881e32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 -- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f918806d390 msgr2=0x7f918813e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f918806d390 0x7f918813e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.420+0000 7f918d000640 1 -- 172.21.15.105:0/414928845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740094d0 con 0x7f91880cbab0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.421+0000 7f918d000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880cbab0 0x7f91881f2840 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f918000c6b0 tx=0x7f918000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.421+0000 7f917fe00640 1 -- 172.21.15.105:0/414928845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180003b70 con 0x7f91880cbab0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.421+0000 7f917fe00640 1 -- 172.21.15.105:0/414928845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180003cf0 con 0x7f91880cbab0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.421+0000 7f917fe00640 1 -- 172.21.15.105:0/414928845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180003ea0 con 0x7f91880cbab0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.421+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91880ae8e0 con 0x7f91880cbab0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.421+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91880aeea0 con 0x7f91880cbab0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.422+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918806ac20 con 0x7f91880cbab0 2024-03-23T07:58:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.423+0000 7f917fe00640 1 -- 172.21.15.105:0/414928845 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9180005310 con 0x7f91880cbab0 2024-03-23T07:58:33.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.424+0000 7f917fe00640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f916806ac00 0x7f916806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.424+0000 7f917fe00640 1 -- 172.21.15.105:0/414928845 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f9180088f30 con 0x7f91880cbab0 2024-03-23T07:58:33.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.424+0000 7f9187e00640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f916806ac00 0x7f916806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.425+0000 7f9187e00640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f916806ac00 0x7f916806d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9178009ad0 tx=0x7f9178008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:33.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.429+0000 7f917fe00640 1 -- 172.21.15.105:0/414928845 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9180054300 con 0x7f91880cbab0 2024-03-23T07:58:33.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.619+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91880adfa0 con 0x7f916806ac00 2024-03-23T07:58:33.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.622+0000 7f917fe00640 1 -- 172.21.15.105:0/414928845 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48413 (secure 0 0 0) 0x7f91780119a0 con 0x7f916806ac00 2024-03-23T07:58:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f916806ac00 msgr2=0x7f916806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f916806ac00 0x7f916806d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9178009ad0 tx=0x7f9178008040 comp rx=0 tx=0).stop 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880cbab0 msgr2=0x7f91881f2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880cbab0 0x7f91881f2840 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f918000c6b0 tx=0x7f918000cb70 comp rx=0 tx=0).stop 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 shutdown_connections 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880cbab0 0x7f91881f2840 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f916806ac00 0x7f916806d0c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91880adbc0 0x7f91881e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 --2- 172.21.15.105:0/414928845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f918806d390 0x7f918813e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.624+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 >> 172.21.15.105:0/414928845 conn(0x7f918806a3d0 msgr2=0x7f9188142bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:33.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.625+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 shutdown_connections 2024-03-23T07:58:33.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.625+0000 7f918f000640 1 -- 172.21.15.105:0/414928845 wait complete. 2024-03-23T07:58:33.645 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-03-23T07:58:32.086781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":221400,"kb_used_data":5244,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753320,"statfs":{"total":773094113280,"available":772867399680,"internally_reserved":0,"allocated":5369856,"data_stored":2994010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16291,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-73,"num_write_kb":-41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001797"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-23T07:58:07.112003+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:58:07.112003+0000","last_peered":"2024-03-23T07:58:07.112003+0000","last_clean":"2024-03-23T07:58:07.112003+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:58:07.112003+0000","last_undegraded":"2024-03-23T07:58:07.112003+0000","last_fullsized":"2024-03-23T07:58:07.112003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476767,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":217089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"min":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3120000000000001,"5min":1.506,"15min":1.506},"last":0.84599999999999997}]},{"osd":1,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.872,"15min":0.88},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3140000000000001,"5min":1.827,"15min":1.827},"last":0.89600000000000002},{"interface":"front","average":{"1min":1,"5min":1.0049999999999999,"15min":1.006},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.46,"5min":1.861,"15min":1.861},"last":1.4039999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.498,"5min":1.944,"15min":1.944},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.093,"15min":1.1040000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.405,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.747}]},{"osd":4,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2490000000000001,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6910000000000001,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.272},{"interface":"front","average":{"1min":1.232,"5min":1.383,"15min":1.4079999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.591,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.056}]},{"osd":5,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.466,"15min":1.498},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.6419999999999999,"5min":3.589,"15min":3.589},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.389,"15min":1.415},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.734,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.546}]},{"osd":6,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.885,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.514,"15min":1.5469999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7130000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.5089999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.389,"15min":1.399},"min":{"1min":0.78100000000000003,"5min":0.753,"15min":0.753},"max":{"1min":1.8160000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7909999999999999,"5min":2.492,"15min":2.492},"last":1.1930000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476767,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":532480,"data_stored":234944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89800000000000002,"15min":0.91400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5229999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.94399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.54700000000000004}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.077,"15min":1.117},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1930000000000001,"5min":2.04,"15min":2.04},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.129,"15min":1.159},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.661,"5min":1.839,"15min":1.839},"last":0.82299999999999995}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.056,"15min":1.085},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7689999999999999,"5min":2.081,"15min":2.081},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.288,"15min":1.331},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.585,"5min":2.012,"15min":2.012},"last":0.95999999999999996}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.4239999999999999,"15min":1.458},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8560000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.129},{"interface":"front","average":{"1min":1.206,"5min":1.3959999999999999,"15min":1.4279999999999999},"min":{"1min":0.67200000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9279999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.93100000000000005}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4610000000000001},"min":{"1min":0.89500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.4990000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.369},{"interface":"front","average":{"1min":1.339,"5min":1.571,"15min":1.609},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9470000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":1.1679999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.504,"15min":1.5449999999999999},"min":{"1min":0.85299999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0249999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.329},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.5},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9610000000000001,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.2609999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.522,"15min":1.556},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.1299999999999999,"5min":3.069,"15min":3.069},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.544,"15min":1.5960000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.0750000000000002,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.4690000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476767,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":807389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3340000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.45,"5min":3.27,"15min":3.27},"last":0.40000000000000002}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.96799999999999997,"15min":0.98099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2110000000000001,"5min":2.758,"15min":2.758},"last":0.309},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0840000000000001,"15min":1.1100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.55,"5min":3.1699999999999999,"15min":3.1699999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.163,"15min":1.1950000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.405,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.1459999999999999,"15min":1.1679999999999999},"min":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3029999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.6080000000000001,"15min":1.6739999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6870000000000001,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.347,"15min":1.377},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5589999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.73299999999999998}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5569999999999999,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.411,"15min":1.4350000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.869,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.88900000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.5580000000000001,"15min":1.6080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6870000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4790000000000001,"15min":1.5129999999999999},"min":{"1min":0.84799999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7829999999999999,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.6140000000000001,"15min":1.6639999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.75,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.875},{"interface":"front","average":{"1min":1.339,"5min":1.512,"15min":1.5409999999999999},"min":{"1min":0.85899999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.9650000000000001,"5min":3.145,"15min":3.145},"last":1.02}]}]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":507904,"data_stored":217091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1359999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.514,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4159999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.622}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.97299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3320000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.995},{"interface":"front","average":{"1min":1.052,"5min":1.153,"15min":1.1699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4970000000000001,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.86299999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.238,"15min":1.244},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7070000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.272},{"interface":"front","average":{"1min":1.149,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.8169999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1739999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.294,"15min":1.304},"min":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9590000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.3440000000000001,"15min":1.359},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9059999999999999,"5min":2.867,"15min":2.867},"last":1.089}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3939999999999999,"15min":1.4019999999999999},"min":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6759999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.3420000000000001,"15min":1.361},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6719999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4690000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.355,"15min":1.373},"min":{"1min":0.89000000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7669999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.52},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3919999999999999,"15min":1.4139999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7130000000000001,"5min":2.411,"15min":2.411},"last":1.415}]}]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":230127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.044,"15min":1.05},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.127},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.107,"15min":1.115},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.351}]},{"osd":1,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2030000000000001,"15min":1.2230000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5189999999999999,"5min":1.893,"15min":1.893},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.1559999999999999,"15min":1.1879999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.091}]},{"osd":2,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.2809999999999999,"15min":1.3120000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5600000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.2989999999999999,"15min":1.3080000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.661,"5min":1.996,"15min":1.996},"last":1.0489999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.581,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.377,"15min":1.403},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6779999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.5269999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.351},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.784,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.391},{"interface":"front","average":{"1min":1.198,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7150000000000001,"5min":2.198,"15min":2.198},"last":1.4259999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.452,"15min":1.468},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.0350000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.51,"15min":1.526},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8420000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.147}]},{"osd":6,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.591,"15min":1.6020000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.008,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.5660000000000001,"15min":1.5820000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.895,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.29}]}]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":0.99199999999999999,"15min":0.97099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0209999999999999,"15min":1.0229999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3660000000000001,"5min":1.387,"15min":1.387},"last":1.1359999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.746,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.776},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.159,"15min":1.1719999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8009999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.97199999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1819999999999999,"15min":1.1839999999999999},"min":{"1min":0.82899999999999996,"5min":0.624,"15min":0.624},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1599999999999999,"15min":1.1519999999999999},"min":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6200000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.548}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":0.97199999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":2.109},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.302,"15min":1.3089999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5589999999999999,"5min":1.968,"15min":1.968},"last":1.379}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.3009999999999999},"min":{"1min":1.0209999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6639999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.716},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.296,"15min":1.284},"min":{"1min":1.0489999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9319999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.196}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.446,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.073,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.075},{"interface":"front","average":{"1min":1.45,"5min":1.466,"15min":1.468},"min":{"1min":1.1799999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":1.804,"5min":2.194,"15min":2.194},"last":1.7789999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4910000000000001,"15min":1.5},"min":{"1min":1.1299999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.72,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.655},{"interface":"front","average":{"1min":1.508,"5min":1.546,"15min":1.552},"min":{"1min":1.2490000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.8899999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.4419999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":230127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.45},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0660000000000001,"15min":1.0669999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.141}]},{"osd":1,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.2989999999999999,"15min":1.339},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5269999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.599},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1759999999999999,"15min":1.194},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2829999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1859999999999999,"15min":1.1970000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5629999999999999,"5min":1.915,"15min":1.915},"last":1.698},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.323,"15min":1.3560000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.478,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.196}]},{"osd":3,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2370000000000001,"15min":1.254},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4379999999999999,"5min":1.734,"15min":1.734},"last":1.018},{"interface":"front","average":{"1min":1.248,"5min":1.387,"15min":1.4099999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.8240000000000001,"5min":1.988,"15min":1.988},"last":1.228}]},{"osd":5,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.347,"15min":1.365},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6379999999999999,"5min":2.198,"15min":2.198},"last":1.339},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.383,"15min":1.413},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.081,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.3759999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.581,"15min":1.613},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9650000000000001,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4630000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5980000000000001,"15min":1.6200000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.121,"5min":2.403,"15min":2.403},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.52,"15min":1.548},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8720000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.504}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":821077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0660000000000001,"15min":1.0920000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.401,"15min":1.401},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.995,"5min":1.097,"15min":1.1140000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.0860000000000001}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1830000000000001,"15min":1.206},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5009999999999999,"5min":1.663,"15min":1.663},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1950000000000001,"15min":1.212},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5580000000000001,"5min":1.833,"15min":1.833},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3120000000000001,"15min":1.3220000000000001},"min":{"1min":0.78000000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.911,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1459999999999999,"15min":1.1619999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5449999999999999,"5min":1.72,"15min":1.72},"last":1.1539999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.254,"15min":1.272},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.583},{"interface":"front","average":{"1min":1.254,"5min":1.3009999999999999,"15min":1.3089999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8220000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3220000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.274,"15min":1.254},"min":{"1min":0.90500000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.484},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.351,"15min":1.369},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.589,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.7}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.456},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.744},{"interface":"front","average":{"1min":1.284,"5min":1.3680000000000001,"15min":1.3819999999999999},"min":{"1min":0.72599999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.724,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.877}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4790000000000001,"15min":1.504},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8580000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.798},{"interface":"front","average":{"1min":1.512,"5min":1.5860000000000001,"15min":1.5980000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1200000000000001,"5min":2.282,"15min":2.282},"last":1.9259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:58:33.646 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-23T07:58:33.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:58:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.853+0000 7f9d0ac00640 1 Processor -- start 2024-03-23T07:58:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.853+0000 7f9d0ac00640 1 -- start start 2024-03-23T07:58:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d0ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415b560 0x7f9d0415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d0ac00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9d0415d410 0x7f9d040b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d0ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0406c6f0 con 0x7f9d040b39b0 2024-03-23T07:58:33.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d0ac00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d040b4370 con 0x7f9d0415d410 2024-03-23T07:58:33.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d0ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0406c9d0 con 0x7f9d0415b560 2024-03-23T07:58:33.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415b560 0x7f9d0415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415b560 0x7f9d0415b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:40704/0 (socket says 172.21.15.105:40704) 2024-03-23T07:58:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.855+0000 7f9d03e00640 1 -- 172.21.15.105:0/2571923215 learned_addr learned my addr 172.21.15.105:0/2571923215 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.856+0000 7f9d08c00640 1 --2- 172.21.15.105:0/2571923215 >> v2:172.21.15.132:3300/0 conn(0x7f9d0415d410 0x7f9d040b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.856+0000 7f9d03e00640 1 -- 172.21.15.105:0/2571923215 >> v2:172.21.15.132:3300/0 conn(0x7f9d0415d410 msgr2=0x7f9d040b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.856+0000 7f9d03e00640 1 --2- 172.21.15.105:0/2571923215 >> v2:172.21.15.132:3300/0 conn(0x7f9d0415d410 0x7f9d040b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.856+0000 7f9d03e00640 1 -- 172.21.15.105:0/2571923215 >> v1:172.21.15.132:6789/0 conn(0x7f9d040b39b0 legacy=0x7f9d0415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.856+0000 7f9d03e00640 1 -- 172.21.15.105:0/2571923215 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0406ccb0 con 0x7f9d0415b560 2024-03-23T07:58:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.858+0000 7f9d03e00640 1 --2- 172.21.15.105:0/2571923215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415b560 0x7f9d0415b940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009870 tx=0x7f9cf402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea00fbd8252c169a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.858+0000 7f9d02a00640 1 -- 172.21.15.105:0/2571923215 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4030840 con 0x7f9d0415b560 2024-03-23T07:58:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.858+0000 7f9d02a00640 1 -- 172.21.15.105:0/2571923215 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf40309a0 con 0x7f9d0415b560 2024-03-23T07:58:33.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.858+0000 7f9d02a00640 1 -- 172.21.15.105:0/2571923215 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4030d30 con 0x7f9d0415b560 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2571923215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415b560 msgr2=0x7f9d0415b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2571923215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415b560 0x7f9d0415b940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009870 tx=0x7f9cf402b980 comp rx=0 tx=0).stop 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2571923215 shutdown_connections 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2571923215 >> v2:172.21.15.132:3300/0 conn(0x7f9d0415d410 0x7f9d040b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2571923215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415b560 0x7f9d0415b940 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2571923215 >> 172.21.15.105:0/2571923215 conn(0x7f9d0406a3d0 msgr2=0x7f9d040b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2571923215 shutdown_connections 2024-03-23T07:58:33.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.859+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2571923215 wait complete. 2024-03-23T07:58:33.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.861+0000 7f9d0ac00640 1 Processor -- start 2024-03-23T07:58:33.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.861+0000 7f9d0ac00640 1 -- start start 2024-03-23T07:58:33.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d0ac00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040b39b0 0x7f9d041f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d0ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d0415b560 0x7f9d040b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d0ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415d410 0x7f9d040afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d0ac00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041f5ca0 con 0x7f9d040b39b0 2024-03-23T07:58:33.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d0ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040b2a00 con 0x7f9d0415b560 2024-03-23T07:58:33.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d0ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040b2ce0 con 0x7f9d0415d410 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040b39b0 0x7f9d041f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040b39b0 0x7f9d041f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43688/0 (socket says 172.21.15.105:43688) 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d03e00640 1 -- 172.21.15.105:0/2714730188 learned_addr learned my addr 172.21.15.105:0/2714730188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d08c00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415d410 0x7f9d040afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.862+0000 7f9d03400640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d0415b560 0x7f9d040b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d03e00640 1 -- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415d410 msgr2=0x7f9d040afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d03e00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415d410 0x7f9d040afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d03e00640 1 -- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d0415b560 msgr2=0x7f9d040b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d03e00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d0415b560 0x7f9d040b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d03e00640 1 -- 172.21.15.105:0/2714730188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf40094d0 con 0x7f9d040b39b0 2024-03-23T07:58:33.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d03400640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d0415b560 0x7f9d040b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:33.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d03e00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040b39b0 0x7f9d041f5760 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:33.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9cfbe00640 1 -- 172.21.15.105:0/2714730188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4039c90 con 0x7f9d040b39b0 2024-03-23T07:58:33.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9cfbe00640 1 -- 172.21.15.105:0/2714730188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf4039df0 con 0x7f9d040b39b0 2024-03-23T07:58:33.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9cfbe00640 1 -- 172.21.15.105:0/2714730188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4043640 con 0x7f9d040b39b0 2024-03-23T07:58:33.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d040b2fc0 con 0x7f9d040b39b0 2024-03-23T07:58:33.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.863+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d040b3580 con 0x7f9d040b39b0 2024-03-23T07:58:33.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.865+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0406ac20 con 0x7f9d040b39b0 2024-03-23T07:58:33.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.865+0000 7f9cfbe00640 1 -- 172.21.15.105:0/2714730188 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9cf40437a0 con 0x7f9d040b39b0 2024-03-23T07:58:33.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.866+0000 7f9cfbe00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ce406ac30 0x7f9ce406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:33.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.866+0000 7f9cfbe00640 1 -- 172.21.15.105:0/2714730188 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f9cf403e070 con 0x7f9d040b39b0 2024-03-23T07:58:33.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.867+0000 7f9d03400640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ce406ac30 0x7f9ce406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:33.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.867+0000 7f9d03400640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ce406ac30 0x7f9ce406d0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d0415ac00 tx=0x7f9cf0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:33.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:33.871+0000 7f9cfbe00640 1 -- 172.21.15.105:0/2714730188 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9cf4077540 con 0x7f9d040b39b0 2024-03-23T07:58:34.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.069+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d0406dd80 con 0x7f9ce406ac30 2024-03-23T07:58:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.075+0000 7f9cfbe00640 1 -- 172.21.15.105:0/2714730188 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48413 (secure 0 0 0) 0x7f9cf00013e0 con 0x7f9ce406ac30 2024-03-23T07:58:34.076 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:34.077 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ce406ac30 msgr2=0x7f9ce406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ce406ac30 0x7f9ce406d0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d0415ac00 tx=0x7f9cf0009040 comp rx=0 tx=0).stop 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040b39b0 msgr2=0x7f9d041f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040b39b0 0x7f9d041f5760 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf4004730 comp rx=0 tx=0).stop 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 shutdown_connections 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ce406ac30 0x7f9ce406d0f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d0415d410 0x7f9d040afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d0415b560 0x7f9d040b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 --2- 172.21.15.105:0/2714730188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040b39b0 0x7f9d041f5760 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 >> 172.21.15.105:0/2714730188 conn(0x7f9d0406a3d0 msgr2=0x7f9d040cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 shutdown_connections 2024-03-23T07:58:34.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.078+0000 7f9d0ac00640 1 -- 172.21.15.105:0/2714730188 wait complete. 2024-03-23T07:58:34.099 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-03-23T07:58:32.086781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":221400,"kb_used_data":5244,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753320,"statfs":{"total":773094113280,"available":772867399680,"internally_reserved":0,"allocated":5369856,"data_stored":2994010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16291,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-73,"num_write_kb":-41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001797"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-23T07:58:07.112003+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:58:07.112003+0000","last_peered":"2024-03-23T07:58:07.112003+0000","last_clean":"2024-03-23T07:58:07.112003+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:58:07.112003+0000","last_undegraded":"2024-03-23T07:58:07.112003+0000","last_fullsized":"2024-03-23T07:58:07.112003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T12:32:26.488103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476767,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":217089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"min":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3120000000000001,"5min":1.506,"15min":1.506},"last":0.84599999999999997}]},{"osd":1,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.872,"15min":0.88},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3140000000000001,"5min":1.827,"15min":1.827},"last":0.89600000000000002},{"interface":"front","average":{"1min":1,"5min":1.0049999999999999,"15min":1.006},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.46,"5min":1.861,"15min":1.861},"last":1.4039999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.498,"5min":1.944,"15min":1.944},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.093,"15min":1.1040000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.405,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.747}]},{"osd":4,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2490000000000001,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6910000000000001,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.272},{"interface":"front","average":{"1min":1.232,"5min":1.383,"15min":1.4079999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.591,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.056}]},{"osd":5,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.466,"15min":1.498},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.6419999999999999,"5min":3.589,"15min":3.589},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.389,"15min":1.415},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.734,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.546}]},{"osd":6,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.885,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.514,"15min":1.5469999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7130000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.5089999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.389,"15min":1.399},"min":{"1min":0.78100000000000003,"5min":0.753,"15min":0.753},"max":{"1min":1.8160000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7909999999999999,"5min":2.492,"15min":2.492},"last":1.1930000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476767,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":532480,"data_stored":234944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89800000000000002,"15min":0.91400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5229999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.94399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.54700000000000004}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.077,"15min":1.117},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1930000000000001,"5min":2.04,"15min":2.04},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.129,"15min":1.159},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.661,"5min":1.839,"15min":1.839},"last":0.82299999999999995}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.056,"15min":1.085},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7689999999999999,"5min":2.081,"15min":2.081},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.288,"15min":1.331},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.585,"5min":2.012,"15min":2.012},"last":0.95999999999999996}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.4239999999999999,"15min":1.458},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8560000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.129},{"interface":"front","average":{"1min":1.206,"5min":1.3959999999999999,"15min":1.4279999999999999},"min":{"1min":0.67200000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9279999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.93100000000000005}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4610000000000001},"min":{"1min":0.89500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.4990000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.369},{"interface":"front","average":{"1min":1.339,"5min":1.571,"15min":1.609},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9470000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":1.1679999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.504,"15min":1.5449999999999999},"min":{"1min":0.85299999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0249999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.329},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.5},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9610000000000001,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.2609999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.522,"15min":1.556},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.1299999999999999,"5min":3.069,"15min":3.069},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.544,"15min":1.5960000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.0750000000000002,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.4690000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476767,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":807389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3340000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.45,"5min":3.27,"15min":3.27},"last":0.40000000000000002}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.96799999999999997,"15min":0.98099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2110000000000001,"5min":2.758,"15min":2.758},"last":0.309},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0840000000000001,"15min":1.1100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.55,"5min":3.1699999999999999,"15min":3.1699999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.163,"15min":1.1950000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.405,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.1459999999999999,"15min":1.1679999999999999},"min":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3029999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.6080000000000001,"15min":1.6739999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6870000000000001,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.347,"15min":1.377},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5589999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.73299999999999998}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5569999999999999,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.411,"15min":1.4350000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.869,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.88900000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.5580000000000001,"15min":1.6080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6870000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4790000000000001,"15min":1.5129999999999999},"min":{"1min":0.84799999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7829999999999999,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.6140000000000001,"15min":1.6639999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.75,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.875},{"interface":"front","average":{"1min":1.339,"5min":1.512,"15min":1.5409999999999999},"min":{"1min":0.85899999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.9650000000000001,"5min":3.145,"15min":3.145},"last":1.02}]}]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":507904,"data_stored":217091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1359999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.514,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4159999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.622}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.97299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3320000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.995},{"interface":"front","average":{"1min":1.052,"5min":1.153,"15min":1.1699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4970000000000001,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.86299999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.238,"15min":1.244},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7070000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.272},{"interface":"front","average":{"1min":1.149,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.8169999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1739999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.294,"15min":1.304},"min":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9590000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.3440000000000001,"15min":1.359},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9059999999999999,"5min":2.867,"15min":2.867},"last":1.089}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3939999999999999,"15min":1.4019999999999999},"min":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6759999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.3420000000000001,"15min":1.361},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6719999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4690000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.355,"15min":1.373},"min":{"1min":0.89000000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7669999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.52},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3919999999999999,"15min":1.4139999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7130000000000001,"5min":2.411,"15min":2.411},"last":1.415}]}]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":230127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.044,"15min":1.05},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.127},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.107,"15min":1.115},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.351}]},{"osd":1,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2030000000000001,"15min":1.2230000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5189999999999999,"5min":1.893,"15min":1.893},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.1559999999999999,"15min":1.1879999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.091}]},{"osd":2,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.2809999999999999,"15min":1.3120000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5600000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.2989999999999999,"15min":1.3080000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.661,"5min":1.996,"15min":1.996},"last":1.0489999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.581,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.377,"15min":1.403},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6779999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.5269999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.351},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.784,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.391},{"interface":"front","average":{"1min":1.198,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7150000000000001,"5min":2.198,"15min":2.198},"last":1.4259999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.452,"15min":1.468},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.0350000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.51,"15min":1.526},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8420000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.147}]},{"osd":6,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.591,"15min":1.6020000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.008,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.5660000000000001,"15min":1.5820000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.895,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.29}]}]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":0.99199999999999999,"15min":0.97099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0209999999999999,"15min":1.0229999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3660000000000001,"5min":1.387,"15min":1.387},"last":1.1359999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.746,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.776},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.159,"15min":1.1719999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8009999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.97199999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1819999999999999,"15min":1.1839999999999999},"min":{"1min":0.82899999999999996,"5min":0.624,"15min":0.624},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1599999999999999,"15min":1.1519999999999999},"min":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6200000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.548}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":0.97199999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":2.109},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.302,"15min":1.3089999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5589999999999999,"5min":1.968,"15min":1.968},"last":1.379}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.3009999999999999},"min":{"1min":1.0209999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6639999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.716},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.296,"15min":1.284},"min":{"1min":1.0489999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9319999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.196}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.446,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.073,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.075},{"interface":"front","average":{"1min":1.45,"5min":1.466,"15min":1.468},"min":{"1min":1.1799999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":1.804,"5min":2.194,"15min":2.194},"last":1.7789999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4910000000000001,"15min":1.5},"min":{"1min":1.1299999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.72,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.655},{"interface":"front","average":{"1min":1.508,"5min":1.546,"15min":1.552},"min":{"1min":1.2490000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.8899999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.4419999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":230127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.45},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0660000000000001,"15min":1.0669999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.141}]},{"osd":1,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.2989999999999999,"15min":1.339},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5269999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.599},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1759999999999999,"15min":1.194},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2829999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1859999999999999,"15min":1.1970000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5629999999999999,"5min":1.915,"15min":1.915},"last":1.698},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.323,"15min":1.3560000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.478,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.196}]},{"osd":3,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2370000000000001,"15min":1.254},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4379999999999999,"5min":1.734,"15min":1.734},"last":1.018},{"interface":"front","average":{"1min":1.248,"5min":1.387,"15min":1.4099999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.8240000000000001,"5min":1.988,"15min":1.988},"last":1.228}]},{"osd":5,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.347,"15min":1.365},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6379999999999999,"5min":2.198,"15min":2.198},"last":1.339},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.383,"15min":1.413},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.081,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.3759999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.581,"15min":1.613},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9650000000000001,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4630000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5980000000000001,"15min":1.6200000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.121,"5min":2.403,"15min":2.403},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.52,"15min":1.548},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8720000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.504}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":821077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0660000000000001,"15min":1.0920000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.401,"15min":1.401},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.995,"5min":1.097,"15min":1.1140000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.0860000000000001}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1830000000000001,"15min":1.206},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5009999999999999,"5min":1.663,"15min":1.663},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1950000000000001,"15min":1.212},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5580000000000001,"5min":1.833,"15min":1.833},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3120000000000001,"15min":1.3220000000000001},"min":{"1min":0.78000000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.911,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1459999999999999,"15min":1.1619999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5449999999999999,"5min":1.72,"15min":1.72},"last":1.1539999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.254,"15min":1.272},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.583},{"interface":"front","average":{"1min":1.254,"5min":1.3009999999999999,"15min":1.3089999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8220000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3220000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.274,"15min":1.254},"min":{"1min":0.90500000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.484},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.351,"15min":1.369},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.589,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.7}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.456},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.744},{"interface":"front","average":{"1min":1.284,"5min":1.3680000000000001,"15min":1.3819999999999999},"min":{"1min":0.72599999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.724,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.877}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4790000000000001,"15min":1.504},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8580000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.798},{"interface":"front","average":{"1min":1.512,"5min":1.5860000000000001,"15min":1.5980000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1200000000000001,"5min":2.282,"15min":2.282},"last":1.9259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:58:34.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T07:58:34.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.312+0000 7f6b23400640 1 Processor -- start 2024-03-23T07:58:34.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.312+0000 7f6b23400640 1 -- start start 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.313+0000 7f6b23400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b1c0b37f0 0x7f6b1c0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b23400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b1c0ccec0 0x7f6b1c06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b23400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1c06da70 con 0x7f6b1c14aa40 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b23400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c06d7d0 con 0x7f6b1c0ccec0 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b23400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b1d00 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b20a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6b1c14aa40 0x7f6b1c0cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43326/0 (socket says 172.21.15.105:43326) 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b20a00640 1 -- 172.21.15.105:0/545920938 learned_addr learned my addr 172.21.15.105:0/545920938 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b21400640 1 --2- 172.21.15.105:0/545920938 >> v2:172.21.15.105:3300/0 conn(0x7f6b1c0ccec0 0x7f6b1c06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b1b400640 1 -- 172.21.15.105:0/545920938 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861484403 0 0) 0x7f6b1c06da70 con 0x7f6b1c14aa40 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b1be00640 1 --2- 172.21.15.105:0/545920938 >> v2:172.21.15.132:3300/0 conn(0x7f6b1c0b37f0 0x7f6b1c0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b1b400640 1 -- 172.21.15.105:0/545920938 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b04003520 con 0x7f6b1c14aa40 2024-03-23T07:58:34.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b21400640 1 -- 172.21.15.105:0/545920938 >> v2:172.21.15.132:3300/0 conn(0x7f6b1c0b37f0 msgr2=0x7f6b1c0cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:34.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b21400640 1 --2- 172.21.15.105:0/545920938 >> v2:172.21.15.132:3300/0 conn(0x7f6b1c0b37f0 0x7f6b1c0cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b21400640 1 -- 172.21.15.105:0/545920938 >> v1:172.21.15.105:6789/0 conn(0x7f6b1c14aa40 legacy=0x7f6b1c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.314+0000 7f6b21400640 1 -- 172.21.15.105:0/545920938 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0b1fe0 con 0x7f6b1c0ccec0 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.315+0000 7f6b21400640 1 --2- 172.21.15.105:0/545920938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0ccec0 0x7f6b1c06c280 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0b3ea0 tx=0x7f6b1402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cae06d537e45ba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b1b400640 1 -- 172.21.15.105:0/545920938 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1402cc80 con 0x7f6b1c0ccec0 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b1b400640 1 -- 172.21.15.105:0/545920938 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1402cde0 con 0x7f6b1c0ccec0 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b1b400640 1 -- 172.21.15.105:0/545920938 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14034660 con 0x7f6b1c0ccec0 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 -- 172.21.15.105:0/545920938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0ccec0 msgr2=0x7f6b1c06c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 --2- 172.21.15.105:0/545920938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0ccec0 0x7f6b1c06c280 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0b3ea0 tx=0x7f6b1402bed0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 -- 172.21.15.105:0/545920938 shutdown_connections 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 --2- 172.21.15.105:0/545920938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0ccec0 0x7f6b1c06c280 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 --2- 172.21.15.105:0/545920938 >> v2:172.21.15.132:3300/0 conn(0x7f6b1c0b37f0 0x7f6b1c0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 -- 172.21.15.105:0/545920938 >> 172.21.15.105:0/545920938 conn(0x7f6b1c06a3d0 msgr2=0x7f6b1c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:34.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 -- 172.21.15.105:0/545920938 shutdown_connections 2024-03-23T07:58:34.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.316+0000 7f6b23400640 1 -- 172.21.15.105:0/545920938 wait complete. 2024-03-23T07:58:34.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.318+0000 7f6b23400640 1 Processor -- start 2024-03-23T07:58:34.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.318+0000 7f6b23400640 1 -- start start 2024-03-23T07:58:34.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.319+0000 7f6b23400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0b37f0 0x7f6b1c0aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:34.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.319+0000 7f6b23400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b1c0ccec0 0x7f6b1c0ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:34.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.319+0000 7f6b23400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b1c14aa40 0x7f6b1c135480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.319+0000 7f6b23400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0ac370 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.319+0000 7f6b23400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0ac650 con 0x7f6b1c0ccec0 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.319+0000 7f6b23400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0ac930 con 0x7f6b1c14aa40 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0b37f0 0x7f6b1c0aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0b37f0 0x7f6b1c0aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43708/0 (socket says 172.21.15.105:43708) 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 -- 172.21.15.105:0/918340402 learned_addr learned my addr 172.21.15.105:0/918340402 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 -- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b1c14aa40 msgr2=0x7f6b1c135480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b21400640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b1c14aa40 0x7f6b1c135480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b1c14aa40 0x7f6b1c135480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b1be00640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b1c0ccec0 0x7f6b1c0ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 -- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b1c0ccec0 msgr2=0x7f6b1c0ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b1c0ccec0 0x7f6b1c0ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 -- 172.21.15.105:0/918340402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b14009a70 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.320+0000 7f6b20a00640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0b37f0 0x7f6b1c0aa2a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000c410 tx=0x7f6b1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.321+0000 7f6b18c00640 1 -- 172.21.15.105:0/918340402 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10005ce0 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.321+0000 7f6b18c00640 1 -- 172.21.15.105:0/918340402 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b10005e40 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.321+0000 7f6b18c00640 1 -- 172.21.15.105:0/918340402 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100072f0 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.321+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c0acc10 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.321+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c0ac020 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.322+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c0ad130 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.323+0000 7f6b18c00640 1 -- 172.21.15.105:0/918340402 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b10007510 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.324+0000 7f6b18c00640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6afc06ac00 0x7f6afc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.324+0000 7f6b18c00640 1 -- 172.21.15.105:0/918340402 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6b10088900 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.324+0000 7f6b1be00640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6afc06ac00 0x7f6afc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:34.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.325+0000 7f6b1be00640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6afc06ac00 0x7f6afc06d0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c164190 tx=0x7f6b08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:34.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.328+0000 7f6b18c00640 1 -- 172.21.15.105:0/918340402 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6b1c0ad130 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.524+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b1c06ac20 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.525+0000 7f6b18c00640 1 -- 172.21.15.105:0/918340402 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f6b10017090 con 0x7f6b1c0b37f0 2024-03-23T07:58:34.526 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"fsid":"e641b836-a0e5-4e95-93aa-bb0628868335","created":"2024-03-23T07:55:53.176457+0000","modified":"2024-03-23T07:58:30.386067+0000","last_up_change":"2024-03-23T07:56:06.383438+0000","last_in_change":"2024-03-23T07:55:56.757504+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T07:56:06.113298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1e926019-a0a8-4e10-b0c3-0fb0c20a45d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":37342},{"type":"v1","addr":"172.21.15.105:6801","nonce":37342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":37342},{"type":"v1","addr":"172.21.15.105:6803","nonce":37342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":37342},{"type":"v1","addr":"172.21.15.105:6807","nonce":37342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":37342},{"type":"v1","addr":"172.21.15.105:6805","nonce":37342}]},"public_addr":"172.21.15.105:6801/37342","cluster_addr":"172.21.15.105:6803/37342","heartbeat_back_addr":"172.21.15.105:6807/37342","heartbeat_front_addr":"172.21.15.105:6805/37342","state":["exists","up"]},{"osd":1,"uuid":"7f82c10f-8491-4200-bf31-48a058dacfa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":37345},{"type":"v1","addr":"172.21.15.105:6814","nonce":37345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":37345},{"type":"v1","addr":"172.21.15.105:6818","nonce":37345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":37345},{"type":"v1","addr":"172.21.15.105:6823","nonce":37345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":37345},{"type":"v1","addr":"172.21.15.105:6821","nonce":37345}]},"public_addr":"172.21.15.105:6814/37345","cluster_addr":"172.21.15.105:6818/37345","heartbeat_back_addr":"172.21.15.105:6823/37345","heartbeat_front_addr":"172.21.15.105:6821/37345","state":["exists","up"]},{"osd":2,"uuid":"c61d9003-c7ec-476e-9bc9-0c1fa7939fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":37346},{"type":"v1","addr":"172.21.15.105:6809","nonce":37346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":37346},{"type":"v1","addr":"172.21.15.105:6811","nonce":37346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":37346},{"type":"v1","addr":"172.21.15.105:6819","nonce":37346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":37346},{"type":"v1","addr":"172.21.15.105:6815","nonce":37346}]},"public_addr":"172.21.15.105:6809/37346","cluster_addr":"172.21.15.105:6811/37346","heartbeat_back_addr":"172.21.15.105:6819/37346","heartbeat_front_addr":"172.21.15.105:6815/37346","state":["exists","up"]},{"osd":3,"uuid":"f3b45192-5bab-4797-9563-205de52e2cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":37347},{"type":"v1","addr":"172.21.15.105:6825","nonce":37347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":37347},{"type":"v1","addr":"172.21.15.105:6827","nonce":37347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":37347},{"type":"v1","addr":"172.21.15.105:6831","nonce":37347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":37347},{"type":"v1","addr":"172.21.15.105:6829","nonce":37347}]},"public_addr":"172.21.15.105:6825/37347","cluster_addr":"172.21.15.105:6827/37347","heartbeat_back_addr":"172.21.15.105:6831/37347","heartbeat_front_addr":"172.21.15.105:6829/37347","state":["exists","up"]},{"osd":4,"uuid":"d1bd131a-59ed-4ac8-b3ff-cd7074eb8387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":37441},{"type":"v1","addr":"172.21.15.132:6801","nonce":37441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":37441},{"type":"v1","addr":"172.21.15.132:6803","nonce":37441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":37441},{"type":"v1","addr":"172.21.15.132:6807","nonce":37441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":37441},{"type":"v1","addr":"172.21.15.132:6805","nonce":37441}]},"public_addr":"172.21.15.132:6801/37441","cluster_addr":"172.21.15.132:6803/37441","heartbeat_back_addr":"172.21.15.132:6807/37441","heartbeat_front_addr":"172.21.15.132:6805/37441","state":["exists","up"]},{"osd":5,"uuid":"5f6cdc52-6519-4e96-be59-77624d9dea1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":37443},{"type":"v1","addr":"172.21.15.132:6819","nonce":37443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":37443},{"type":"v1","addr":"172.21.15.132:6823","nonce":37443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":37443},{"type":"v1","addr":"172.21.15.132:6831","nonce":37443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":37443},{"type":"v1","addr":"172.21.15.132:6828","nonce":37443}]},"public_addr":"172.21.15.132:6819/37443","cluster_addr":"172.21.15.132:6823/37443","heartbeat_back_addr":"172.21.15.132:6831/37443","heartbeat_front_addr":"172.21.15.132:6828/37443","state":["exists","up"]},{"osd":6,"uuid":"bb44da31-9fd8-4442-bdb0-0b1cdae07bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":37442},{"type":"v1","addr":"172.21.15.132:6809","nonce":37442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":37442},{"type":"v1","addr":"172.21.15.132:6811","nonce":37442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":37442},{"type":"v1","addr":"172.21.15.132:6815","nonce":37442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":37442},{"type":"v1","addr":"172.21.15.132:6813","nonce":37442}]},"public_addr":"172.21.15.132:6809/37442","cluster_addr":"172.21.15.132:6811/37442","heartbeat_back_addr":"172.21.15.132:6815/37442","heartbeat_front_addr":"172.21.15.132:6813/37442","state":["exists","up"]},{"osd":7,"uuid":"a643e4ae-7365-4306-8e2e-1502e320da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":37452},{"type":"v1","addr":"172.21.15.132:6818","nonce":37452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":37452},{"type":"v1","addr":"172.21.15.132:6822","nonce":37452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":37452},{"type":"v1","addr":"172.21.15.132:6829","nonce":37452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":37452},{"type":"v1","addr":"172.21.15.132:6826","nonce":37452}]},"public_addr":"172.21.15.132:6818/37452","cluster_addr":"172.21.15.132:6822/37452","heartbeat_back_addr":"172.21.15.132:6829/37452","heartbeat_front_addr":"172.21.15.132:6826/37452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.282145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.062683+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.086379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.288973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:01.892821+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.084197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T07:56:02.128908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6835/2670581804":"2024-03-24T07:58:27.149583+0000","172.21.15.105:6834/2670581804":"2024-03-24T07:58:27.149583+0000","172.21.15.105:6837/3723879365":"2024-03-24T07:58:14.253538+0000","172.21.15.105:6836/3723879365":"2024-03-24T07:58:14.253538+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T07:58:34.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6afc06ac00 msgr2=0x7f6afc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6afc06ac00 0x7f6afc06d0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c164190 tx=0x7f6b08008040 comp rx=0 tx=0).stop 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0b37f0 msgr2=0x7f6b1c0aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0b37f0 0x7f6b1c0aa2a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000c410 tx=0x7f6b1000c8d0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 shutdown_connections 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6afc06ac00 0x7f6afc06d0c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b1c14aa40 0x7f6b1c135480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b1c0ccec0 0x7f6b1c0ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 --2- 172.21.15.105:0/918340402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1c0b37f0 0x7f6b1c0aa2a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 >> 172.21.15.105:0/918340402 conn(0x7f6b1c06a3d0 msgr2=0x7f6b1c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 shutdown_connections 2024-03-23T07:58:34.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:34.528+0000 7f6b23400640 1 -- 172.21.15.105:0/918340402 wait complete. 2024-03-23T07:58:35.547 INFO:tasks.ceph:Scrubbing osd.0 2024-03-23T07:58:35.547 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:35.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.769+0000 7f6526200640 1 Processor -- start 2024-03-23T07:58:35.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.769+0000 7f6526200640 1 -- start start 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6526200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f652015a850 0x7f65200cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6526200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65200b22b0 0x7f652015eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6526200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f652015f020 0x7f65200cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6526200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652015ac30 con 0x7f652015a850 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6526200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652015e320 con 0x7f652015f020 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6526200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cda20 con 0x7f65200b22b0 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6524e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f652015a850 0x7f65200cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6524e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f652015a850 0x7f65200cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43714/0 (socket says 172.21.15.105:43714) 2024-03-23T07:58:35.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.771+0000 7f6524e00640 1 -- 172.21.15.105:0/4169974655 learned_addr learned my addr 172.21.15.105:0/4169974655 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:35.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f651fe00640 1 --2- 172.21.15.105:0/4169974655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65200b22b0 0x7f652015eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f6524e00640 1 -- 172.21.15.105:0/4169974655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65200b22b0 msgr2=0x7f652015eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:35.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f6525800640 1 --2- 172.21.15.105:0/4169974655 >> v2:172.21.15.132:3300/0 conn(0x7f652015f020 0x7f65200cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f6524e00640 1 --2- 172.21.15.105:0/4169974655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65200b22b0 0x7f652015eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f6524e00640 1 -- 172.21.15.105:0/4169974655 >> v2:172.21.15.132:3300/0 conn(0x7f652015f020 msgr2=0x7f65200cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f6524e00640 1 --2- 172.21.15.105:0/4169974655 >> v2:172.21.15.132:3300/0 conn(0x7f652015f020 0x7f65200cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f6524e00640 1 -- 172.21.15.105:0/4169974655 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200cdd00 con 0x7f652015a850 2024-03-23T07:58:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.772+0000 7f651fe00640 1 --2- 172.21.15.105:0/4169974655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65200b22b0 0x7f652015eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:35.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.773+0000 7f6524e00640 1 --2- 172.21.15.105:0/4169974655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200cd1d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f652006bb00 tx=0x7f651002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51df8be40f35767c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:35.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.773+0000 7f651f400640 1 -- 172.21.15.105:0/4169974655 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651002ca70 con 0x7f652015a850 2024-03-23T07:58:35.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.773+0000 7f651f400640 1 -- 172.21.15.105:0/4169974655 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651002cbd0 con 0x7f652015a850 2024-03-23T07:58:35.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.773+0000 7f651f400640 1 -- 172.21.15.105:0/4169974655 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65100343f0 con 0x7f652015a850 2024-03-23T07:58:35.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 -- 172.21.15.105:0/4169974655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 msgr2=0x7f65200cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:35.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 --2- 172.21.15.105:0/4169974655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200cd1d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f652006bb00 tx=0x7f651002bed0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 -- 172.21.15.105:0/4169974655 shutdown_connections 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 --2- 172.21.15.105:0/4169974655 >> v2:172.21.15.132:3300/0 conn(0x7f652015f020 0x7f65200cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 --2- 172.21.15.105:0/4169974655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65200b22b0 0x7f652015eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 --2- 172.21.15.105:0/4169974655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200cd1d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 -- 172.21.15.105:0/4169974655 >> 172.21.15.105:0/4169974655 conn(0x7f65200b01b0 msgr2=0x7f65200b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 -- 172.21.15.105:0/4169974655 shutdown_connections 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.774+0000 7f6526200640 1 -- 172.21.15.105:0/4169974655 wait complete. 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.776+0000 7f6526200640 1 Processor -- start 2024-03-23T07:58:35.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.776+0000 7f6526200640 1 -- start start 2024-03-23T07:58:35.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f6526200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65200b22b0 0x7f6520154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f6526200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f6526200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f652015f020 0x7f65200c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f6526200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f2f70 con 0x7f652015a850 2024-03-23T07:58:35.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f6526200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f59b0 con 0x7f65200b22b0 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f6526200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f5c90 con 0x7f652015f020 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f651fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f651fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43726/0 (socket says 172.21.15.105:43726) 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.777+0000 7f651fe00640 1 -- 172.21.15.105:0/912120024 learned_addr learned my addr 172.21.15.105:0/912120024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f6524e00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65200b22b0 0x7f6520154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f6525800640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f652015f020 0x7f65200c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f651fe00640 1 -- 172.21.15.105:0/912120024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f652015f020 msgr2=0x7f65200c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f651fe00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f652015f020 0x7f65200c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f651fe00640 1 -- 172.21.15.105:0/912120024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65200b22b0 msgr2=0x7f6520154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f651fe00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65200b22b0 0x7f6520154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f651fe00640 1 -- 172.21.15.105:0/912120024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510009b30 con 0x7f652015a850 2024-03-23T07:58:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f651fe00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200c7650 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c7c0 tx=0x7f650c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.778+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c007a30 con 0x7f652015a850 2024-03-23T07:58:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.779+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c007b90 con 0x7f652015a850 2024-03-23T07:58:35.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.779+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65201f5f70 con 0x7f652015a850 2024-03-23T07:58:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.779+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c00ce70 con 0x7f652015a850 2024-03-23T07:58:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.779+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65201f6490 con 0x7f652015a850 2024-03-23T07:58:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.780+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65201f5b20 con 0x7f652015a850 2024-03-23T07:58:35.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.781+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f650c0113f0 con 0x7f652015a850 2024-03-23T07:58:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.781+0000 7f651cc00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f650006ac00 0x7f650006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.782+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f650c0893b0 con 0x7f652015a850 2024-03-23T07:58:35.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.782+0000 7f6524e00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f650006ac00 0x7f650006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.782+0000 7f651cc00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f6500070fa0 0x7f65000733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.782+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 --> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6500073ab0 con 0x7f6500070fa0 2024-03-23T07:58:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.782+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650c08b990 con 0x7f652015a850 2024-03-23T07:58:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.782+0000 7f6525800640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f6500070fa0 0x7f65000733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.782+0000 7f6524e00640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f650006ac00 0x7f650006d0c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6510035000 tx=0x7f6510030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:35.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.783+0000 7f6525800640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f6500070fa0 0x7f65000733e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:35.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.783+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== osd.0 v2:172.21.15.105:6800/37342 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6500073ab0 con 0x7f6500070fa0 2024-03-23T07:58:35.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.797+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 --> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65201f5e00 con 0x7f6500070fa0 2024-03-23T07:58:35.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f651cc00640 1 -- 172.21.15.105:0/912120024 <== osd.0 v2:172.21.15.105:6800/37342 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f65201f5e00 con 0x7f6500070fa0 2024-03-23T07:58:35.798 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:35.798 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:35.799 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:35.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f6500070fa0 msgr2=0x7f65000733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f6526200640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6800/37342,v1:172.21.15.105:6801/37342] conn(0x7f6500070fa0 0x7f65000733e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f650006ac00 msgr2=0x7f650006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f6526200640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f650006ac00 0x7f650006d0c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6510035000 tx=0x7f6510030000 comp rx=0 tx=0).stop 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 msgr2=0x7f65200c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f6526200640 1 --2- 172.21.15.105:0/912120024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652015a850 0x7f65200c7650 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c7c0 tx=0x7f650c00cc80 comp rx=0 tx=0).stop 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.798+0000 7f6525800640 1 -- 172.21.15.105:0/912120024 reap_dead start 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.799+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 shutdown_connections 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.799+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 >> 172.21.15.105:0/912120024 conn(0x7f65200b01b0 msgr2=0x7f6520157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.799+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 shutdown_connections 2024-03-23T07:58:35.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:35.799+0000 7f6526200640 1 -- 172.21.15.105:0/912120024 wait complete. 2024-03-23T07:58:35.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-23T07:58:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.011+0000 7ff7d3200640 1 Processor -- start 2024-03-23T07:58:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.011+0000 7ff7d3200640 1 -- start start 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.012+0000 7ff7d3200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7d4140cf0 0x7ff7d40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.012+0000 7ff7d3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff7d40b23e0 0x7ff7d40cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.012+0000 7ff7d3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d40cb6b0 con 0x7ff7d40cb2b0 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.012+0000 7ff7d3200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d41410d0 con 0x7ff7d4140cf0 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.012+0000 7ff7d3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d40cd1b0 con 0x7ff7d40b23e0 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.013+0000 7ff7d1e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7d4140cf0 0x7ff7d40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.013+0000 7ff7d1e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7d4140cf0 0x7ff7d40b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43736/0 (socket says 172.21.15.105:43736) 2024-03-23T07:58:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.013+0000 7ff7d1e00640 1 -- 172.21.15.105:0/4027185535 learned_addr learned my addr 172.21.15.105:0/4027185535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.013+0000 7ff7d1e00640 1 -- 172.21.15.105:0/4027185535 >> v2:172.21.15.132:3300/0 conn(0x7ff7d40b23e0 msgr2=0x7ff7d40cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.013+0000 7ff7d1e00640 1 --2- 172.21.15.105:0/4027185535 >> v2:172.21.15.132:3300/0 conn(0x7ff7d40b23e0 0x7ff7d40cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.013+0000 7ff7d1e00640 1 -- 172.21.15.105:0/4027185535 >> v1:172.21.15.132:6789/0 conn(0x7ff7d40cb2b0 legacy=0x7ff7d4070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.013+0000 7ff7d1e00640 1 -- 172.21.15.105:0/4027185535 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40cd490 con 0x7ff7d4140cf0 2024-03-23T07:58:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.014+0000 7ff7d1e00640 1 --2- 172.21.15.105:0/4027185535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40b1ea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40cc010 tx=0x7ff7c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a79d6526024109 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.014+0000 7ff7d0a00640 1 -- 172.21.15.105:0/4027185535 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4031870 con 0x7ff7d4140cf0 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.014+0000 7ff7d0a00640 1 -- 172.21.15.105:0/4027185535 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c40319d0 con 0x7ff7d4140cf0 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.014+0000 7ff7d0a00640 1 -- 172.21.15.105:0/4027185535 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4031d60 con 0x7ff7d4140cf0 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 -- 172.21.15.105:0/4027185535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 msgr2=0x7ff7d40b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 --2- 172.21.15.105:0/4027185535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40b1ea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40cc010 tx=0x7ff7c402b910 comp rx=0 tx=0).stop 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 -- 172.21.15.105:0/4027185535 shutdown_connections 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 --2- 172.21.15.105:0/4027185535 >> v2:172.21.15.132:3300/0 conn(0x7ff7d40b23e0 0x7ff7d40cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 --2- 172.21.15.105:0/4027185535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40b1ea0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 -- 172.21.15.105:0/4027185535 >> 172.21.15.105:0/4027185535 conn(0x7ff7d4065e20 msgr2=0x7ff7d4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 -- 172.21.15.105:0/4027185535 shutdown_connections 2024-03-23T07:58:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.015+0000 7ff7d3200640 1 -- 172.21.15.105:0/4027185535 wait complete. 2024-03-23T07:58:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.017+0000 7ff7d3200640 1 Processor -- start 2024-03-23T07:58:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.017+0000 7ff7d3200640 1 -- start start 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7d40b23e0 0x7ff7d41ef680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7d40cb2b0 0x7ff7d415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40a7c70 con 0x7ff7d4140cf0 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40a8230 con 0x7ff7d40cb2b0 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d41efbc0 con 0x7ff7d40b23e0 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d2800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d2800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40aa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43752/0 (socket says 172.21.15.105:43752) 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.018+0000 7ff7d2800640 1 -- 172.21.15.105:0/2507743600 learned_addr learned my addr 172.21.15.105:0/2507743600 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d1e00640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7d40b23e0 0x7ff7d41ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d2800640 1 -- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7d40b23e0 msgr2=0x7ff7d41ef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d1400640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7d40cb2b0 0x7ff7d415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d2800640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7d40b23e0 0x7ff7d41ef680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d2800640 1 -- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7d40cb2b0 msgr2=0x7ff7d415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d2800640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7d40cb2b0 0x7ff7d415c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d2800640 1 -- 172.21.15.105:0/2507743600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40094d0 con 0x7ff7d4140cf0 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d2800640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40aa630 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c780 tx=0x7ff7cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7ca000640 1 -- 172.21.15.105:0/2507743600 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc003d80 con 0x7ff7d4140cf0 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7ca000640 1 -- 172.21.15.105:0/2507743600 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00ce90 con 0x7ff7d4140cf0 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d415d240 con 0x7ff7d4140cf0 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7ca000640 1 -- 172.21.15.105:0/2507743600 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0052b0 con 0x7ff7d4140cf0 2024-03-23T07:58:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.019+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d40a7ec0 con 0x7ff7d4140cf0 2024-03-23T07:58:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.020+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d415d760 con 0x7ff7d4140cf0 2024-03-23T07:58:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.021+0000 7ff7ca000640 1 -- 172.21.15.105:0/2507743600 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff7cc005410 con 0x7ff7d4140cf0 2024-03-23T07:58:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.022+0000 7ff7ca000640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff7b406ac00 0x7ff7b406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.022+0000 7ff7ca000640 1 -- 172.21.15.105:0/2507743600 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ff7cc088ad0 con 0x7ff7d4140cf0 2024-03-23T07:58:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.023+0000 7ff7d1e00640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff7b406ac00 0x7ff7b406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.023+0000 7ff7d1e00640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff7b406ac00 0x7ff7b406d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4036000 tx=0x7ff7c403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.027+0000 7ff7ca000640 1 -- 172.21.15.105:0/2507743600 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff7cc053d70 con 0x7ff7d4140cf0 2024-03-23T07:58:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.221+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7d40cda50 con 0x7ff7b406ac00 2024-03-23T07:58:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.222+0000 7ff7ca000640 1 -- 172.21.15.105:0/2507743600 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff7c4031d10 con 0x7ff7b406ac00 2024-03-23T07:58:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 0 to deep-scrub 2024-03-23T07:58:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff7b406ac00 msgr2=0x7ff7b406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff7b406ac00 0x7ff7b406d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4036000 tx=0x7ff7c403c040 comp rx=0 tx=0).stop 2024-03-23T07:58:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 msgr2=0x7ff7d40aa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40aa630 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c780 tx=0x7ff7cc00cc40 comp rx=0 tx=0).stop 2024-03-23T07:58:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 shutdown_connections 2024-03-23T07:58:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7d4140cf0 0x7ff7d40aa630 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7d40cb2b0 0x7ff7d415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff7b406ac00 0x7ff7b406d0c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 --2- 172.21.15.105:0/2507743600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7d40b23e0 0x7ff7d41ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.224+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 >> 172.21.15.105:0/2507743600 conn(0x7ff7d4065e20 msgr2=0x7ff7d40b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.225+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 shutdown_connections 2024-03-23T07:58:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.225+0000 7ff7d3200640 1 -- 172.21.15.105:0/2507743600 wait complete. 2024-03-23T07:58:36.244 INFO:tasks.ceph:Scrubbing osd.1 2024-03-23T07:58:36.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:36.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.450+0000 7f3764400640 1 Processor -- start 2024-03-23T07:58:36.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.450+0000 7f3764400640 1 -- start start 2024-03-23T07:58:36.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3764400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f375c0cd740 0x7f375c0b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3764400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f375c0b5aa0 0x7f375c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3764400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c0d20c0 con 0x7f375c0d0d40 2024-03-23T07:58:36.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3764400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c0d14b0 con 0x7f375c0cd740 2024-03-23T07:58:36.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3764400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c0d1790 con 0x7f375c0b5aa0 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3761a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f375c0cd740 0x7f375c0b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3761000640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f375c0d0d40 0x7f375c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43330/0 (socket says 172.21.15.105:43330) 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3761000640 1 -- 172.21.15.105:0/2132943427 learned_addr learned my addr 172.21.15.105:0/2132943427 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3761a00640 1 -- 172.21.15.105:0/2132943427 >> v2:172.21.15.132:3300/0 conn(0x7f375c0b5aa0 msgr2=0x7f375c0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3761a00640 1 --2- 172.21.15.105:0/2132943427 >> v2:172.21.15.132:3300/0 conn(0x7f375c0b5aa0 0x7f375c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.452+0000 7f3761a00640 1 -- 172.21.15.105:0/2132943427 >> v1:172.21.15.105:6789/0 conn(0x7f375c0d0d40 legacy=0x7f375c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.453+0000 7f3761a00640 1 -- 172.21.15.105:0/2132943427 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c0d1a70 con 0x7f375c0cd740 2024-03-23T07:58:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.453+0000 7f375be00640 1 -- 172.21.15.105:0/2132943427 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890930281 0 0) 0x7f375c0d20c0 con 0x7f375c0d0d40 2024-03-23T07:58:36.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.454+0000 7f3761a00640 1 --2- 172.21.15.105:0/2132943427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0cd740 0x7f375c0b84b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f375c0c99c0 tx=0x7f375402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7896adb1f6a61e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.454+0000 7f375be00640 1 -- 172.21.15.105:0/2132943427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754031830 con 0x7f375c0cd740 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.454+0000 7f375be00640 1 -- 172.21.15.105:0/2132943427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3754031990 con 0x7f375c0cd740 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.454+0000 7f375be00640 1 -- 172.21.15.105:0/2132943427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754031d20 con 0x7f375c0cd740 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 -- 172.21.15.105:0/2132943427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0cd740 msgr2=0x7f375c0b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 --2- 172.21.15.105:0/2132943427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0cd740 0x7f375c0b84b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f375c0c99c0 tx=0x7f375402b8d0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 -- 172.21.15.105:0/2132943427 shutdown_connections 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 --2- 172.21.15.105:0/2132943427 >> v2:172.21.15.132:3300/0 conn(0x7f375c0b5aa0 0x7f375c0b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 --2- 172.21.15.105:0/2132943427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0cd740 0x7f375c0b84b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 -- 172.21.15.105:0/2132943427 >> 172.21.15.105:0/2132943427 conn(0x7f375c0cf5d0 msgr2=0x7f375c0cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 -- 172.21.15.105:0/2132943427 shutdown_connections 2024-03-23T07:58:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.455+0000 7f3764400640 1 -- 172.21.15.105:0/2132943427 wait complete. 2024-03-23T07:58:36.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.457+0000 7f3764400640 1 Processor -- start 2024-03-23T07:58:36.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.457+0000 7f3764400640 1 -- start start 2024-03-23T07:58:36.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3764400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0b5aa0 0x7f375c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3764400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f375c0cd740 0x7f375c1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3764400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f375c0d0d40 0x7f375c1f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3764400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1598a0 con 0x7f375c0b5aa0 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3764400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c158a30 con 0x7f375c0d0d40 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3764400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c158c90 con 0x7f375c0cd740 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0b5aa0 0x7f375c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0b5aa0 0x7f375c0b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43774/0 (socket says 172.21.15.105:43774) 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 -- 172.21.15.105:0/1785766776 learned_addr learned my addr 172.21.15.105:0/1785766776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 -- 172.21.15.105:0/1785766776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f375c0cd740 msgr2=0x7f375c1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f375c0cd740 0x7f375c1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 -- 172.21.15.105:0/1785766776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f375c0d0d40 msgr2=0x7f375c1f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f375c0d0d40 0x7f375c1f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.458+0000 7f3761a00640 1 -- 172.21.15.105:0/1785766776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3754009400 con 0x7f375c0b5aa0 2024-03-23T07:58:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.459+0000 7f3761a00640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0b5aa0 0x7f375c0b4de0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3754009890 tx=0x7f37540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.459+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375403ba70 con 0x7f375c0b5aa0 2024-03-23T07:58:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.459+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375403bbf0 con 0x7f375c0b5aa0 2024-03-23T07:58:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.459+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37540453f0 con 0x7f375c0b5aa0 2024-03-23T07:58:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.459+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c158f70 con 0x7f375c0b5aa0 2024-03-23T07:58:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.459+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c159530 con 0x7f375c0b5aa0 2024-03-23T07:58:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.460+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f375c160300 con 0x7f375c0b5aa0 2024-03-23T07:58:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.461+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3754045550 con 0x7f375c0b5aa0 2024-03-23T07:58:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.462+0000 7f3759600640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f373c06ac00 0x7f373c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.462+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f37540ac1e0 con 0x7f375c0b5aa0 2024-03-23T07:58:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.462+0000 7f3759600640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f373c070fa0 0x7f373c0733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.462+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 --> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f373c073ab0 con 0x7f373c070fa0 2024-03-23T07:58:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.462+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37540ac550 con 0x7f375c0b5aa0 2024-03-23T07:58:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.462+0000 7f3761000640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f373c06ac00 0x7f373c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.462+0000 7f3762400640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f373c070fa0 0x7f373c0733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.463+0000 7f3761000640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f373c06ac00 0x7f373c06d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3748007fd0 tx=0x7f3748007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.463+0000 7f3762400640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f373c070fa0 0x7f373c0733e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.464+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== osd.1 v2:172.21.15.105:6812/37345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f373c073ab0 con 0x7f373c070fa0 2024-03-23T07:58:36.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.477+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 --> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f375c0d11d0 con 0x7f373c070fa0 2024-03-23T07:58:36.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3759600640 1 -- 172.21.15.105:0/1785766776 <== osd.1 v2:172.21.15.105:6812/37345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f374c0097c0 con 0x7f373c070fa0 2024-03-23T07:58:36.478 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:36.479 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f373c070fa0 msgr2=0x7f373c0733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6812/37345,v1:172.21.15.105:6814/37345] conn(0x7f373c070fa0 0x7f373c0733e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f373c06ac00 msgr2=0x7f373c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f373c06ac00 0x7f373c06d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3748007fd0 tx=0x7f3748007ce0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0b5aa0 msgr2=0x7f375c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 --2- 172.21.15.105:0/1785766776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f375c0b5aa0 0x7f375c0b4de0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3754009890 tx=0x7f37540046c0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3762400640 1 -- 172.21.15.105:0/1785766776 reap_dead start 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 shutdown_connections 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.478+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 >> 172.21.15.105:0/1785766776 conn(0x7f375c0cf5d0 msgr2=0x7f375c15d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.479+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 shutdown_connections 2024-03-23T07:58:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.479+0000 7f3764400640 1 -- 172.21.15.105:0/1785766776 wait complete. 2024-03-23T07:58:36.496 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-23T07:58:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.701+0000 7f0299c00640 1 Processor -- start 2024-03-23T07:58:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.701+0000 7f0299c00640 1 -- start start 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0299c00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f029415c710 0x7f029415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0299c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0294070730 0x7f02940cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0299c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029415d100 con 0x7f02940cda00 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0299c00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029415d3e0 con 0x7f029415c710 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0299c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029415d6c0 con 0x7f0294070730 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0293e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f029415c710 0x7f029415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0293e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f029415c710 0x7f029415caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43788/0 (socket says 172.21.15.105:43788) 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.703+0000 7f0293e00640 1 -- 172.21.15.105:0/433332200 learned_addr learned my addr 172.21.15.105:0/433332200 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.704+0000 7f0299200640 1 --2- 172.21.15.105:0/433332200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0294070730 0x7f02940cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.704+0000 7f0293e00640 1 -- 172.21.15.105:0/433332200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0294070730 msgr2=0x7f02940cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:36.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.704+0000 7f0293e00640 1 --2- 172.21.15.105:0/433332200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0294070730 0x7f02940cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.704+0000 7f0293e00640 1 -- 172.21.15.105:0/433332200 >> v1:172.21.15.132:6789/0 conn(0x7f02940cda00 legacy=0x7f0294070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:36.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.704+0000 7f0293e00640 1 -- 172.21.15.105:0/433332200 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029415d9a0 con 0x7f029415c710 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.705+0000 7f0293e00640 1 --2- 172.21.15.105:0/433332200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f029415c710 0x7f029415caf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0284009870 tx=0x7f028402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47076e7d55b13696 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.705+0000 7f0292a00640 1 -- 172.21.15.105:0/433332200 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284031640 con 0x7f029415c710 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0292a00640 1 -- 172.21.15.105:0/433332200 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02840317a0 con 0x7f029415c710 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0292a00640 1 -- 172.21.15.105:0/433332200 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284031b30 con 0x7f029415c710 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 -- 172.21.15.105:0/433332200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f029415c710 msgr2=0x7f029415caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 --2- 172.21.15.105:0/433332200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f029415c710 0x7f029415caf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0284009870 tx=0x7f028402b8f0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 -- 172.21.15.105:0/433332200 shutdown_connections 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 --2- 172.21.15.105:0/433332200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0294070730 0x7f02940cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 --2- 172.21.15.105:0/433332200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f029415c710 0x7f029415caf0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 -- 172.21.15.105:0/433332200 >> 172.21.15.105:0/433332200 conn(0x7f0294065e20 msgr2=0x7f0294069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 -- 172.21.15.105:0/433332200 shutdown_connections 2024-03-23T07:58:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.706+0000 7f0299c00640 1 -- 172.21.15.105:0/433332200 wait complete. 2024-03-23T07:58:36.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.708+0000 7f0299c00640 1 Processor -- start 2024-03-23T07:58:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.708+0000 7f0299c00640 1 -- start start 2024-03-23T07:58:36.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0299c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0294070730 0x7f0294152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0299c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cda00 0x7f029406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0299c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f029415c710 0x7f029406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0299c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294153000 con 0x7f02940cda00 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0299c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02940a8cc0 con 0x7f0294070730 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0299c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940a8700 con 0x7f029415c710 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0293400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cda00 0x7f029406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0293400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cda00 0x7f029406c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43804/0 (socket says 172.21.15.105:43804) 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.709+0000 7f0293400640 1 -- 172.21.15.105:0/187415054 learned_addr learned my addr 172.21.15.105:0/187415054 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293e00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0294070730 0x7f0294152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293400640 1 -- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f029415c710 msgr2=0x7f029406cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293400640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f029415c710 0x7f029406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293400640 1 -- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0294070730 msgr2=0x7f0294152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293400640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0294070730 0x7f0294152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293400640 1 -- 172.21.15.105:0/187415054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02840094d0 con 0x7f02940cda00 2024-03-23T07:58:36.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293e00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0294070730 0x7f0294152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:36.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f0293400640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cda00 0x7f029406c590 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f028000c430 tx=0x7f028000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.710+0000 7f028be00640 1 -- 172.21.15.105:0/187415054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280011790 con 0x7f02940cda00 2024-03-23T07:58:36.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.711+0000 7f028be00640 1 -- 172.21.15.105:0/187415054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02800118f0 con 0x7f02940cda00 2024-03-23T07:58:36.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.711+0000 7f028be00640 1 -- 172.21.15.105:0/187415054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280011ce0 con 0x7f02940cda00 2024-03-23T07:58:36.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.711+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02940a89e0 con 0x7f02940cda00 2024-03-23T07:58:36.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.711+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029415a9d0 con 0x7f02940cda00 2024-03-23T07:58:36.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.712+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029415a550 con 0x7f02940cda00 2024-03-23T07:58:36.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.713+0000 7f028be00640 1 -- 172.21.15.105:0/187415054 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f028001c460 con 0x7f02940cda00 2024-03-23T07:58:36.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.714+0000 7f028be00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f027406ac00 0x7f027406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:36.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.714+0000 7f028be00640 1 -- 172.21.15.105:0/187415054 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f0280089160 con 0x7f02940cda00 2024-03-23T07:58:36.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.714+0000 7f0293e00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f027406ac00 0x7f027406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:36.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.715+0000 7f0293e00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f027406ac00 0x7f027406d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0284036000 tx=0x7f028403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:36.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.718+0000 7f028be00640 1 -- 172.21.15.105:0/187415054 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f029415a550 con 0x7f02940cda00 2024-03-23T07:58:36.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.913+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f029415a550 con 0x7f027406ac00 2024-03-23T07:58:36.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.914+0000 7f028be00640 1 -- 172.21.15.105:0/187415054 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f029415a550 con 0x7f027406ac00 2024-03-23T07:58:36.916 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 1 to deep-scrub 2024-03-23T07:58:36.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f027406ac00 msgr2=0x7f027406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f027406ac00 0x7f027406d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0284036000 tx=0x7f028403c040 comp rx=0 tx=0).stop 2024-03-23T07:58:36.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cda00 msgr2=0x7f029406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:36.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cda00 0x7f029406c590 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f028000c430 tx=0x7f028000c8f0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 shutdown_connections 2024-03-23T07:58:36.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f027406ac00 0x7f027406d0c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f029415c710 0x7f029406cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cda00 0x7f029406c590 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 --2- 172.21.15.105:0/187415054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0294070730 0x7f0294152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:36.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 >> 172.21.15.105:0/187415054 conn(0x7f0294065e20 msgr2=0x7f0294160e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:36.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 shutdown_connections 2024-03-23T07:58:36.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:36.917+0000 7f0299c00640 1 -- 172.21.15.105:0/187415054 wait complete. 2024-03-23T07:58:36.935 INFO:tasks.ceph:Scrubbing osd.2 2024-03-23T07:58:36.936 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.153+0000 7fbab9400640 1 Processor -- start 2024-03-23T07:58:37.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.153+0000 7fbab9400640 1 -- start start 2024-03-23T07:58:37.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab9400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbab4070cd0 0x7fbab4160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab9400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab406d390 con 0x7fbab40cf040 2024-03-23T07:58:37.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab9400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab406d670 con 0x7fbab41456a0 2024-03-23T07:58:37.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab9400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab40ad890 con 0x7fbab4070cd0 2024-03-23T07:58:37.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab1e00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbab40cf040 0x7fbab41604c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59494/0 (socket says 172.21.15.105:59494) 2024-03-23T07:58:37.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab1e00640 1 -- 172.21.15.105:0/2601289340 learned_addr learned my addr 172.21.15.105:0/2601289340 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:37.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427495269 0 0) 0x7fbab406d390 con 0x7fbab40cf040 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba98003520 con 0x7fbab40cf040 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.155+0000 7fbab3200640 1 --2- 172.21.15.105:0/2601289340 >> v2:172.21.15.132:3300/0 conn(0x7fbab4070cd0 0x7fbab4160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2516397109 0 0) 0x7fba98003520 con 0x7fbab40cf040 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 >> v2:172.21.15.132:3300/0 conn(0x7fbab4070cd0 msgr2=0x7fbab4160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 --2- 172.21.15.105:0/2601289340 >> v2:172.21.15.132:3300/0 conn(0x7fbab4070cd0 0x7fbab4160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 >> v1:172.21.15.132:6789/0 conn(0x7fbab41456a0 legacy=0x7fbab40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40701f0 con 0x7fbab40cf040 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fbaa0001670 con 0x7fbab40cf040 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 >> v1:172.21.15.105:6789/0 conn(0x7fbab40cf040 legacy=0x7fbab41604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba980039c0 0x7fba98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba980047c0 0x7fba98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba980051d0 0x7fba9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40ad890 con 0x7fba980051d0 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9800c120 con 0x7fba980047c0 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9800c2e0 con 0x7fba980039c0 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.156+0000 7fbab3200640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba980051d0 0x7fba9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.157+0000 7fbab3200640 1 -- 172.21.15.105:0/2601289340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba980039c0 msgr2=0x7fba98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.157+0000 7fbab3200640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba980039c0 0x7fba98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.157+0000 7fbab3200640 1 -- 172.21.15.105:0/2601289340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba980047c0 msgr2=0x7fba98004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.157+0000 7fbab3200640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba980047c0 0x7fba98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.157+0000 7fbab3200640 1 -- 172.21.15.105:0/2601289340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba98003520 con 0x7fba980051d0 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.158+0000 7fbab3200640 1 -- 172.21.15.105:0/2601289340 reap_dead start 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.158+0000 7fbab3200640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba980051d0 0x7fba9800b980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbaac007360 tx=0x7fba980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c284d854351cdc4c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.158+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac02ccf0 con 0x7fba980051d0 2024-03-23T07:58:37.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.158+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac02ce90 con 0x7fba980051d0 2024-03-23T07:58:37.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.158+0000 7fbab1400640 1 -- 172.21.15.105:0/2601289340 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac003bd0 con 0x7fba980051d0 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.159+0000 7fbab9400640 1 -- 172.21.15.105:0/2601289340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba980051d0 msgr2=0x7fba9800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.159+0000 7fbab9400640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba980051d0 0x7fba9800b980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbaac007360 tx=0x7fba980057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.159+0000 7fbab9400640 1 -- 172.21.15.105:0/2601289340 shutdown_connections 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.159+0000 7fbab9400640 1 --2- 172.21.15.105:0/2601289340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba980051d0 0x7fba9800b980 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.159+0000 7fbab9400640 1 -- 172.21.15.105:0/2601289340 >> 172.21.15.105:0/2601289340 conn(0x7fbab406a0b0 msgr2=0x7fbab406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.159+0000 7fbab9400640 1 -- 172.21.15.105:0/2601289340 shutdown_connections 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.159+0000 7fbab9400640 1 -- 172.21.15.105:0/2601289340 wait complete. 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.161+0000 7fbab9400640 1 Processor -- start 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.161+0000 7fbab9400640 1 -- start start 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.161+0000 7fbab9400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba980051d0 0x7fbab40cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab9400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbab40c6070 0x7fbab40c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab415de30 0x7fbab415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab9400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41f22e0 con 0x7fbab415de30 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab9400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40c6a30 con 0x7fba980051d0 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab9400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab415f670 con 0x7fbab40c6070 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab415de30 0x7fbab415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab415de30 0x7fbab415e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42726/0 (socket says 172.21.15.105:42726) 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 -- 172.21.15.105:0/1157763206 learned_addr learned my addr 172.21.15.105:0/1157763206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 -- 172.21.15.105:0/1157763206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbab40c6070 msgr2=0x7fbab40c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbab40c6070 0x7fbab40c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 -- 172.21.15.105:0/1157763206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba980051d0 msgr2=0x7fbab40cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab2800640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba980051d0 0x7fbab40cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba980051d0 0x7fbab40cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.162+0000 7fbab3200640 1 -- 172.21.15.105:0/1157763206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab415e7f0 con 0x7fbab415de30 2024-03-23T07:58:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.163+0000 7fbab3200640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab415de30 0x7fbab415e2b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbaac007480 tx=0x7fbaac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.163+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac004780 con 0x7fbab415de30 2024-03-23T07:58:37.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.163+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac003e30 con 0x7fbab415de30 2024-03-23T07:58:37.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.163+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab415ea60 con 0x7fbab415de30 2024-03-23T07:58:37.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.163+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac03d690 con 0x7fbab415de30 2024-03-23T07:58:37.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.163+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab415f020 con 0x7fbab415de30 2024-03-23T07:58:37.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.164+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbab40707b0 con 0x7fbab415de30 2024-03-23T07:58:37.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.165+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbaac034440 con 0x7fbab415de30 2024-03-23T07:58:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.166+0000 7fbaaaa00640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fba9406ac00 0x7fba9406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.166+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fbaac0aa9f0 con 0x7fbab415de30 2024-03-23T07:58:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.166+0000 7fbab2800640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fba9406ac00 0x7fba9406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.166+0000 7fbaaaa00640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7fba94070fa0 0x7fba940733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.166+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 --> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba94073ab0 con 0x7fba94070fa0 2024-03-23T07:58:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.166+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaac0af2a0 con 0x7fbab415de30 2024-03-23T07:58:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.167+0000 7fbab1e00640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7fba94070fa0 0x7fba940733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.167+0000 7fbab2800640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fba9406ac00 0x7fba9406d0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000f20 tx=0x7fbaa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.167+0000 7fbab1e00640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7fba94070fa0 0x7fba940733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.168+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== osd.2 v2:172.21.15.105:6808/37346 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fba94073ab0 con 0x7fba94070fa0 2024-03-23T07:58:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.181+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 --> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbab4145a80 con 0x7fba94070fa0 2024-03-23T07:58:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.182+0000 7fbaaaa00640 1 -- 172.21.15.105:0/1157763206 <== osd.2 v2:172.21.15.105:6808/37346 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbaa0007a40 con 0x7fba94070fa0 2024-03-23T07:58:37.183 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:37.183 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:37.183 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.182+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7fba94070fa0 msgr2=0x7fba940733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.182+0000 7fbab9400640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6808/37346,v1:172.21.15.105:6809/37346] conn(0x7fba94070fa0 0x7fba940733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.182+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fba9406ac00 msgr2=0x7fba9406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.182+0000 7fbab9400640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fba9406ac00 0x7fba9406d0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000f20 tx=0x7fbaa4009040 comp rx=0 tx=0).stop 2024-03-23T07:58:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.182+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab415de30 msgr2=0x7fbab415e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.182+0000 7fbab9400640 1 --2- 172.21.15.105:0/1157763206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab415de30 0x7fbab415e2b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbaac007480 tx=0x7fbaac004ab0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.183+0000 7fbab3200640 1 -- 172.21.15.105:0/1157763206 reap_dead start 2024-03-23T07:58:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.183+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 shutdown_connections 2024-03-23T07:58:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.183+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 >> 172.21.15.105:0/1157763206 conn(0x7fbab406a0b0 msgr2=0x7fbab40af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.183+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 shutdown_connections 2024-03-23T07:58:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.183+0000 7fbab9400640 1 -- 172.21.15.105:0/1157763206 wait complete. 2024-03-23T07:58:37.202 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-23T07:58:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.402+0000 7fa30a000640 1 Processor -- start 2024-03-23T07:58:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.402+0000 7fa30a000640 1 -- start start 2024-03-23T07:58:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.403+0000 7fa30a000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa304159690 0x7fa3040ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.403+0000 7fa30a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3040cd890 0x7fa3040cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.403+0000 7fa30a000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30415e010 con 0x7fa3040cc330 2024-03-23T07:58:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.403+0000 7fa30a000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3040ce1f0 con 0x7fa304159690 2024-03-23T07:58:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.403+0000 7fa30a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040cc7f0 con 0x7fa3040cd890 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa2faa00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa3040cc330 0x7fa3040b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59498/0 (socket says 172.21.15.105:59498) 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa2faa00640 1 -- 172.21.15.105:0/3173600921 learned_addr learned my addr 172.21.15.105:0/3173600921 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667492235 0 0) 0x7fa30415e010 con 0x7fa3040cc330 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa303400640 1 --2- 172.21.15.105:0/3173600921 >> v2:172.21.15.132:3300/0 conn(0x7fa304159690 0x7fa3040ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa303e00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3040cd890 0x7fa3040cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e4003520 con 0x7fa3040cc330 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 648271541 0 0) 0x7fa2e4003520 con 0x7fa3040cc330 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3040cd890 msgr2=0x7fa3040cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3040cd890 0x7fa3040cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 >> v2:172.21.15.132:3300/0 conn(0x7fa304159690 msgr2=0x7fa3040ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 --2- 172.21.15.105:0/3173600921 >> v2:172.21.15.132:3300/0 conn(0x7fa304159690 0x7fa3040ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040ccad0 con 0x7fa3040cc330 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fa2ec001670 con 0x7fa3040cc330 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 >> v1:172.21.15.105:6789/0 conn(0x7fa3040cc330 legacy=0x7fa3040b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2e4003970 0x7fa2e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.404+0000 7fa302a00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2e40047c0 0x7fa2e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa302a00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa2e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040cc7f0 con 0x7fa2e4005180 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e400bf70 con 0x7fa2e4003970 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e400c100 con 0x7fa2e40047c0 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303e00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3040cd890 0x7fa3040cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303400640 1 --2- 172.21.15.105:0/3173600921 >> v2:172.21.15.132:3300/0 conn(0x7fa304159690 0x7fa3040ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303e00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa2e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303e00640 1 -- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2e40047c0 msgr2=0x7fa2e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303e00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2e40047c0 0x7fa2e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303e00640 1 -- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2e4003970 msgr2=0x7fa2e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303400640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2e4003970 0x7fa2e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303e00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2e4003970 0x7fa2e4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.405+0000 7fa303e00640 1 -- 172.21.15.105:0/3173600921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4003520 con 0x7fa2e4005180 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.406+0000 7fa303e00640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa2e400b980 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009f30 tx=0x7fa2e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc63d3eb8a5b1d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc02cc70 con 0x7fa2e4005180 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc02cdd0 con 0x7fa2e4005180 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa302a00640 1 -- 172.21.15.105:0/3173600921 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc003bb0 con 0x7fa2e4005180 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa30a000640 1 -- 172.21.15.105:0/3173600921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 msgr2=0x7fa2e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa30a000640 1 --2- 172.21.15.105:0/3173600921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa2e400b980 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009f30 tx=0x7fa2e40057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa303e00640 1 -- 172.21.15.105:0/3173600921 reap_dead start 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa30a000640 1 -- 172.21.15.105:0/3173600921 shutdown_connections 2024-03-23T07:58:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa30a000640 1 -- 172.21.15.105:0/3173600921 >> 172.21.15.105:0/3173600921 conn(0x7fa30406a0b0 msgr2=0x7fa30406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa30a000640 1 -- 172.21.15.105:0/3173600921 shutdown_connections 2024-03-23T07:58:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.407+0000 7fa30a000640 1 -- 172.21.15.105:0/3173600921 wait complete. 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.409+0000 7fa30a000640 1 Processor -- start 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.409+0000 7fa30a000640 1 -- start start 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa30a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa3041441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa30a000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa304146ca0 0x7fa3041577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa30a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa304157d20 0x7fa304153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa30a000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa304158200 con 0x7fa2e4005180 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa30a000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3041584e0 con 0x7fa304146ca0 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa30a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3041587c0 con 0x7fa304157d20 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa303400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa3041441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa303400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa3041441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42748/0 (socket says 172.21.15.105:42748) 2024-03-23T07:58:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.410+0000 7fa303400640 1 -- 172.21.15.105:0/4121984374 learned_addr learned my addr 172.21.15.105:0/4121984374 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa303400640 1 -- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa304157d20 msgr2=0x7fa304153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa303400640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa304157d20 0x7fa304153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa2faa00640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa304146ca0 0x7fa3041577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa303400640 1 -- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa304146ca0 msgr2=0x7fa3041577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa303400640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa304146ca0 0x7fa3041577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa303400640 1 -- 172.21.15.105:0/4121984374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304158aa0 con 0x7fa2e4005180 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa2faa00640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa304146ca0 0x7fa3041577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa303400640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa3041441b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400f4f0 tx=0x7fa2f400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa2fbe00640 1 -- 172.21.15.105:0/4121984374 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4014890 con 0x7fa2e4005180 2024-03-23T07:58:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa2fbe00640 1 -- 172.21.15.105:0/4121984374 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f400c030 con 0x7fa2e4005180 2024-03-23T07:58:37.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa2fbe00640 1 -- 172.21.15.105:0/4121984374 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4014c30 con 0x7fa2e4005180 2024-03-23T07:58:37.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.411+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3041446f0 con 0x7fa2e4005180 2024-03-23T07:58:37.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.412+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041544c0 con 0x7fa2e4005180 2024-03-23T07:58:37.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.413+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa304073390 con 0x7fa2e4005180 2024-03-23T07:58:37.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.413+0000 7fa2fbe00640 1 -- 172.21.15.105:0/4121984374 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa2f40133e0 con 0x7fa2e4005180 2024-03-23T07:58:37.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.415+0000 7fa2fbe00640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa2e006ac00 0x7fa2e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.415+0000 7fa2fbe00640 1 -- 172.21.15.105:0/4121984374 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fa2f40896b0 con 0x7fa2e4005180 2024-03-23T07:58:37.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.415+0000 7fa2faa00640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa2e006ac00 0x7fa2e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.416+0000 7fa2faa00640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa2e006ac00 0x7fa2e006d0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000ff0 tx=0x7fa2ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.419+0000 7fa2fbe00640 1 -- 172.21.15.105:0/4121984374 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa2f4054950 con 0x7fa2e4005180 2024-03-23T07:58:37.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.613+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa304158650 con 0x7fa2e006ac00 2024-03-23T07:58:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.614+0000 7fa2fbe00640 1 -- 172.21.15.105:0/4121984374 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa304158650 con 0x7fa2e006ac00 2024-03-23T07:58:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 2 to deep-scrub 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa2e006ac00 msgr2=0x7fa2e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa2e006ac00 0x7fa2e006d0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000ff0 tx=0x7fa2ec009040 comp rx=0 tx=0).stop 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 msgr2=0x7fa3041441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa3041441b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400f4f0 tx=0x7fa2f400f9b0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 shutdown_connections 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fa2e006ac00 0x7fa2e006d0c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa304157d20 0x7fa304153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa304146ca0 0x7fa3041577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 --2- 172.21.15.105:0/4121984374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e4005180 0x7fa3041441b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 >> 172.21.15.105:0/4121984374 conn(0x7fa30406a0b0 msgr2=0x7fa3040abe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 shutdown_connections 2024-03-23T07:58:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.617+0000 7fa30a000640 1 -- 172.21.15.105:0/4121984374 wait complete. 2024-03-23T07:58:37.636 INFO:tasks.ceph:Scrubbing osd.3 2024-03-23T07:58:37.637 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:37.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.852+0000 7f74af400640 1 Processor -- start 2024-03-23T07:58:37.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.852+0000 7f74af400640 1 -- start start 2024-03-23T07:58:37.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.854+0000 7f74af400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74a806ce90 0x7f74a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.854+0000 7f74af400640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a80cdba0 con 0x7f74a80ce930 2024-03-23T07:58:37.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.854+0000 7f74af400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a80cde80 con 0x7f74a80add30 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.854+0000 7f74af400640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a80ae1c0 con 0x7f74a806ce90 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.854+0000 7f74ad400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74a806ce90 0x7f74a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.854+0000 7f74ad400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74a806ce90 0x7f74a806d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42754/0 (socket says 172.21.15.105:42754) 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.854+0000 7f74ad400640 1 -- 172.21.15.105:0/747513453 learned_addr learned my addr 172.21.15.105:0/747513453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.855+0000 7f74ad400640 1 -- 172.21.15.105:0/747513453 >> v1:172.21.15.132:6789/0 conn(0x7f74a80add30 legacy=0x7f74a80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.855+0000 7f74ad400640 1 -- 172.21.15.105:0/747513453 >> v1:172.21.15.105:6789/0 conn(0x7f74a80ce930 legacy=0x7f74a80cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.855+0000 7f74a7400640 1 -- 172.21.15.105:0/747513453 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072203049 0 0) 0x7f74a80cdba0 con 0x7f74a80ce930 2024-03-23T07:58:37.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.855+0000 7f74ad400640 1 -- 172.21.15.105:0/747513453 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80ce160 con 0x7f74a806ce90 2024-03-23T07:58:37.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.856+0000 7f74ad400640 1 --2- 172.21.15.105:0/747513453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a806ce90 0x7f74a806d300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009e50 tx=0x7f74a002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcc879b5a8634e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.856+0000 7f74a7400640 1 -- 172.21.15.105:0/747513453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a002cb20 con 0x7f74a806ce90 2024-03-23T07:58:37.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.856+0000 7f74a7400640 1 -- 172.21.15.105:0/747513453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a002cc80 con 0x7f74a806ce90 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.856+0000 7f74a7400640 1 -- 172.21.15.105:0/747513453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a00343f0 con 0x7f74a806ce90 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.857+0000 7f74af400640 1 -- 172.21.15.105:0/747513453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a806ce90 msgr2=0x7f74a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.857+0000 7f74af400640 1 --2- 172.21.15.105:0/747513453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a806ce90 0x7f74a806d300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009e50 tx=0x7f74a002bf80 comp rx=0 tx=0).stop 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.857+0000 7f74af400640 1 -- 172.21.15.105:0/747513453 shutdown_connections 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.857+0000 7f74af400640 1 --2- 172.21.15.105:0/747513453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a806ce90 0x7f74a806d300 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.857+0000 7f74af400640 1 -- 172.21.15.105:0/747513453 >> 172.21.15.105:0/747513453 conn(0x7f74a806a0b0 msgr2=0x7f74a806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.857+0000 7f74af400640 1 -- 172.21.15.105:0/747513453 shutdown_connections 2024-03-23T07:58:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.857+0000 7f74af400640 1 -- 172.21.15.105:0/747513453 wait complete. 2024-03-23T07:58:37.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.859+0000 7f74af400640 1 Processor -- start 2024-03-23T07:58:37.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.859+0000 7f74af400640 1 -- start start 2024-03-23T07:58:37.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74af400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a806ce90 0x7f74a8150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74af400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a80add30 0x7f74a81513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74af400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74a80ce930 0x7f74a814be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74af400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80afe10 con 0x7f74a80add30 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74af400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a815b9e0 con 0x7f74a806ce90 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74af400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80af570 con 0x7f74a80ce930 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a80add30 0x7f74a81513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a80add30 0x7f74a81513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42768/0 (socket says 172.21.15.105:42768) 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 -- 172.21.15.105:0/2073513356 learned_addr learned my addr 172.21.15.105:0/2073513356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 -- 172.21.15.105:0/2073513356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74a80ce930 msgr2=0x7f74a814be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74a80ce930 0x7f74a814be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 -- 172.21.15.105:0/2073513356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a806ce90 msgr2=0x7f74a8150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a806ce90 0x7f74a8150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.860+0000 7f74a7e00640 1 -- 172.21.15.105:0/2073513356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a0009b30 con 0x7f74a80add30 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.861+0000 7f74a7e00640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a80add30 0x7f74a81513c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f749400c460 tx=0x7f749400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.861+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494012830 con 0x7f74a80add30 2024-03-23T07:58:37.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.861+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74940129d0 con 0x7f74a80add30 2024-03-23T07:58:37.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.861+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a80af850 con 0x7f74a80add30 2024-03-23T07:58:37.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.861+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494012d80 con 0x7f74a80add30 2024-03-23T07:58:37.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.861+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a8151900 con 0x7f74a80add30 2024-03-23T07:58:37.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.862+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74a815df90 con 0x7f74a80add30 2024-03-23T07:58:37.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.863+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f749401b400 con 0x7f74a80add30 2024-03-23T07:58:37.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.864+0000 7f74a4c00640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f748806ac00 0x7f748806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.864+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f7494089140 con 0x7f74a80add30 2024-03-23T07:58:37.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.864+0000 7f74aca00640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f748806ac00 0x7f748806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.864+0000 7f74a4c00640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7f7488070fa0 0x7f74880733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:37.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.864+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 --> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7488073ab0 con 0x7f7488070fa0 2024-03-23T07:58:37.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.864+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f749408b800 con 0x7f74a80add30 2024-03-23T07:58:37.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.865+0000 7f74ad400640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7f7488070fa0 0x7f74880733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:37.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.865+0000 7f74aca00640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f748806ac00 0x7f748806d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f749c001560 tx=0x7f749c007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.865+0000 7f74ad400640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7f7488070fa0 0x7f74880733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:37.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.866+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== osd.3 v2:172.21.15.105:6824/37347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7488073ab0 con 0x7f7488070fa0 2024-03-23T07:58:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.880+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 --> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f74a80b42d0 con 0x7f7488070fa0 2024-03-23T07:58:37.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74a4c00640 1 -- 172.21.15.105:0/2073513356 <== osd.3 v2:172.21.15.105:6824/37347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f74a002c9a0 con 0x7f7488070fa0 2024-03-23T07:58:37.881 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:37.881 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:37.881 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:37.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7f7488070fa0 msgr2=0x7f74880733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74af400640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6824/37347,v1:172.21.15.105:6825/37347] conn(0x7f7488070fa0 0x7f74880733e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f748806ac00 msgr2=0x7f748806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74af400640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f748806ac00 0x7f748806d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f749c001560 tx=0x7f749c007930 comp rx=0 tx=0).stop 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a80add30 msgr2=0x7f74a81513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74af400640 1 --2- 172.21.15.105:0/2073513356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74a80add30 0x7f74a81513c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f749400c460 tx=0x7f749400c920 comp rx=0 tx=0).stop 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74ad400640 1 -- 172.21.15.105:0/2073513356 reap_dead start 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.881+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 shutdown_connections 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.882+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 >> 172.21.15.105:0/2073513356 conn(0x7f74a806a0b0 msgr2=0x7f74a80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.882+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 shutdown_connections 2024-03-23T07:58:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:37.882+0000 7f74af400640 1 -- 172.21.15.105:0/2073513356 wait complete. 2024-03-23T07:58:37.898 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-23T07:58:38.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.111+0000 7f7fa0a00640 1 Processor -- start 2024-03-23T07:58:38.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.111+0000 7f7fa0a00640 1 -- start start 2024-03-23T07:58:38.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7fa0a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c15c820 0x7f7f9c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7fa0a00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7f9c0cf7c0 0x7f7f9c0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7fa0a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c0ccd90 con 0x7f7f9c160e90 2024-03-23T07:58:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7fa0a00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cdd70 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7fa0a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0b7fc0 con 0x7f7f9c15c820 2024-03-23T07:58:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7f92a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7f9c160e90 0x7f7f9c0cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59514/0 (socket says 172.21.15.105:59514) 2024-03-23T07:58:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7f92a00640 1 -- 172.21.15.105:0/1919775686 learned_addr learned my addr 172.21.15.105:0/1919775686 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7f9be00640 1 --2- 172.21.15.105:0/1919775686 >> v2:172.21.15.105:3300/0 conn(0x7f7f9c0cf7c0 0x7f7f9c0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.113+0000 7f7f9b400640 1 --2- 172.21.15.105:0/1919775686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c15c820 0x7f7f9c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.114+0000 7f7f9be00640 1 -- 172.21.15.105:0/1919775686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c15c820 msgr2=0x7f7f9c0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.114+0000 7f7f9be00640 1 --2- 172.21.15.105:0/1919775686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c15c820 0x7f7f9c0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.114+0000 7f7f9be00640 1 -- 172.21.15.105:0/1919775686 >> v1:172.21.15.105:6789/0 conn(0x7f7f9c160e90 legacy=0x7f7f9c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.114+0000 7f7f9be00640 1 -- 172.21.15.105:0/1919775686 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0b82a0 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.114+0000 7f7f9aa00640 1 -- 172.21.15.105:0/1919775686 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897692768 0 0) 0x7f7f9c0ccd90 con 0x7f7f9c160e90 2024-03-23T07:58:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.115+0000 7f7f9be00640 1 --2- 172.21.15.105:0/1919775686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0d0560 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097a0 tx=0x7f7f9402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766dcfd3ed195101 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.115+0000 7f7f9aa00640 1 -- 172.21.15.105:0/1919775686 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94030800 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.115+0000 7f7f9aa00640 1 -- 172.21.15.105:0/1919775686 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94030960 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.115+0000 7f7f9aa00640 1 -- 172.21.15.105:0/1919775686 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94030cf0 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 -- 172.21.15.105:0/1919775686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 msgr2=0x7f7f9c0d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/1919775686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0d0560 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097a0 tx=0x7f7f9402b940 comp rx=0 tx=0).stop 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 -- 172.21.15.105:0/1919775686 shutdown_connections 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/1919775686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0d0560 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/1919775686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c15c820 0x7f7f9c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 -- 172.21.15.105:0/1919775686 >> 172.21.15.105:0/1919775686 conn(0x7f7f9c06b070 msgr2=0x7f7f9c06b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 -- 172.21.15.105:0/1919775686 shutdown_connections 2024-03-23T07:58:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.116+0000 7f7fa0a00640 1 -- 172.21.15.105:0/1919775686 wait complete. 2024-03-23T07:58:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.117+0000 7f7fa0a00640 1 Processor -- start 2024-03-23T07:58:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.118+0000 7f7fa0a00640 1 -- start start 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.118+0000 7f7fa0a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0a9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7fa0a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f9c15c820 0x7f7f9c1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7fa0a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c160e90 0x7f7f9c1f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7fa0a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c15d2d0 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7fa0a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c15d520 con 0x7f7f9c15c820 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7fa0a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c15d800 con 0x7f7f9c160e90 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0a9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42786/0 (socket says 172.21.15.105:42786) 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 -- 172.21.15.105:0/2638374814 learned_addr learned my addr 172.21.15.105:0/2638374814 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 -- 172.21.15.105:0/2638374814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c160e90 msgr2=0x7f7f9c1f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c160e90 0x7f7f9c1f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 -- 172.21.15.105:0/2638374814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f9c15c820 msgr2=0x7f7f9c1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f9c15c820 0x7f7f9c1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.119+0000 7f7f9b400640 1 -- 172.21.15.105:0/2638374814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f94009400 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.120+0000 7f7f9b400640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0a9300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c007ef0 tx=0x7f7f8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.120+0000 7f7f93e00640 1 -- 172.21.15.105:0/2638374814 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c014620 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.120+0000 7f7f93e00640 1 -- 172.21.15.105:0/2638374814 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c014780 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.120+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c15dae0 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.120+0000 7f7f93e00640 1 -- 172.21.15.105:0/2638374814 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c014b30 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.120+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c0a9840 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.121+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c0d2480 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.122+0000 7f7f93e00640 1 -- 172.21.15.105:0/2638374814 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7f8c014c90 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.123+0000 7f7f93e00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7f7806ac00 0x7f7f7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.123+0000 7f7f93e00640 1 -- 172.21.15.105:0/2638374814 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f7f8c089c00 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.123+0000 7f7f92a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7f7806ac00 0x7f7f7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.124+0000 7f7f92a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7f7806ac00 0x7f7f7806d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7f84007fd0 tx=0x7f7f84007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.128+0000 7f7f93e00640 1 -- 172.21.15.105:0/2638374814 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7f8c054f30 con 0x7f7f9c0cf7c0 2024-03-23T07:58:38.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.324+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f9c0b8580 con 0x7f7f7806ac00 2024-03-23T07:58:38.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.325+0000 7f7f93e00640 1 -- 172.21.15.105:0/2638374814 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7f840070d0 con 0x7f7f7806ac00 2024-03-23T07:58:38.325 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 3 to deep-scrub 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7f7806ac00 msgr2=0x7f7f7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7f7806ac00 0x7f7f7806d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7f84007fd0 tx=0x7f7f84007960 comp rx=0 tx=0).stop 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 msgr2=0x7f7f9c0a9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0a9300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c007ef0 tx=0x7f7f8c00e4b0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 shutdown_connections 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7f7806ac00 0x7f7f7806d0c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f9c160e90 0x7f7f9c1f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f9c15c820 0x7f7f9c1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 --2- 172.21.15.105:0/2638374814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f9c0cf7c0 0x7f7f9c0a9300 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 >> 172.21.15.105:0/2638374814 conn(0x7f7f9c06b070 msgr2=0x7f7f9c0ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:38.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 shutdown_connections 2024-03-23T07:58:38.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.328+0000 7f7fa0a00640 1 -- 172.21.15.105:0/2638374814 wait complete. 2024-03-23T07:58:38.349 INFO:tasks.ceph:Scrubbing osd.4 2024-03-23T07:58:38.349 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:38.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.563+0000 7efcc1200640 1 Processor -- start 2024-03-23T07:58:38.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.563+0000 7efcc1200640 1 -- start start 2024-03-23T07:58:38.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.565+0000 7efcc1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc159830 0x7efcbc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.565+0000 7efcc1200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efcbc0ce930 0x7efcbc0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.565+0000 7efcc1200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc159c10 con 0x7efcbc0ca7b0 2024-03-23T07:58:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.565+0000 7efcc1200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc06de50 con 0x7efcbc0ce930 2024-03-23T07:58:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.565+0000 7efcc1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0cf300 con 0x7efcbc159830 2024-03-23T07:58:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcb9e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc159830 0x7efcbc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcb9e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc159830 0x7efcbc0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.105:47186/0 (socket says 172.21.15.105:47186) 2024-03-23T07:58:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcb9e00640 1 -- 172.21.15.105:0/692227589 learned_addr learned my addr 172.21.15.105:0/692227589 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcba800640 1 --2- 172.21.15.105:0/692227589 >> v2:172.21.15.132:3300/0 conn(0x7efcbc0ce930 0x7efcbc0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcba800640 1 -- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc159830 msgr2=0x7efcbc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:38.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcba800640 1 --2- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc159830 0x7efcbc0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcba800640 1 -- 172.21.15.105:0/692227589 >> v1:172.21.15.132:6789/0 conn(0x7efcbc0ca7b0 legacy=0x7efcbc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcba800640 1 -- 172.21.15.105:0/692227589 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0cd840 con 0x7efcbc0ce930 2024-03-23T07:58:38.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.566+0000 7efcb9e00640 1 --2- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc159830 0x7efcbc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:58:38.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.568+0000 7efcba800640 1 --2- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc0ce930 0x7efcbc0cedc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009d90 tx=0x7efcb402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1624b6b730440a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.569+0000 7efcb9400640 1 -- 172.21.15.105:0/692227589 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb402cce0 con 0x7efcbc0ce930 2024-03-23T07:58:38.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.569+0000 7efcb9400640 1 -- 172.21.15.105:0/692227589 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb402ce40 con 0x7efcbc0ce930 2024-03-23T07:58:38.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.569+0000 7efcb9400640 1 -- 172.21.15.105:0/692227589 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4034660 con 0x7efcbc0ce930 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 -- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc0ce930 msgr2=0x7efcbc0cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 --2- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc0ce930 0x7efcbc0cedc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009d90 tx=0x7efcb402bf30 comp rx=0 tx=0).stop 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 -- 172.21.15.105:0/692227589 shutdown_connections 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 --2- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc0ce930 0x7efcbc0cedc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 --2- 172.21.15.105:0/692227589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc159830 0x7efcbc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 -- 172.21.15.105:0/692227589 >> 172.21.15.105:0/692227589 conn(0x7efcbc06a0b0 msgr2=0x7efcbc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 -- 172.21.15.105:0/692227589 shutdown_connections 2024-03-23T07:58:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.570+0000 7efcc1200640 1 -- 172.21.15.105:0/692227589 wait complete. 2024-03-23T07:58:38.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.572+0000 7efcc1200640 1 Processor -- start 2024-03-23T07:58:38.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.572+0000 7efcc1200640 1 -- start start 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcc1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc0ca7b0 0x7efcbc1f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcc1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0ce930 0x7efcbc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcc1200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc159830 0x7efcbc15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcc1200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0afca0 con 0x7efcbc0ce930 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcc1200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b07c0 con 0x7efcbc159830 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcc1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0af090 con 0x7efcbc0ca7b0 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcb1400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0ce930 0x7efcbc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcb1400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0ce930 0x7efcbc15c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42792/0 (socket says 172.21.15.105:42792) 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.573+0000 7efcb1400640 1 -- 172.21.15.105:0/1279399583 learned_addr learned my addr 172.21.15.105:0/1279399583 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb9e00640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc0ca7b0 0x7efcbc1f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcba800640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc159830 0x7efcbc15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb1400640 1 -- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc0ca7b0 msgr2=0x7efcbc1f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb1400640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcbc0ca7b0 0x7efcbc1f13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb1400640 1 -- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc159830 msgr2=0x7efcbc15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb1400640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcbc159830 0x7efcbc15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb1400640 1 -- 172.21.15.105:0/1279399583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac00b770 con 0x7efcbc0ce930 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb1400640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0ce930 0x7efcbc15c0b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efca400c450 tx=0x7efca400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40052d0 con 0x7efcbc0ce930 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca4005430 con 0x7efcbc0ce930 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4012870 con 0x7efcbc0ce930 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.574+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb4009a70 con 0x7efcbc0ce930 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.575+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc0af650 con 0x7efcbc0ce930 2024-03-23T07:58:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.576+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efcbc0cd1f0 con 0x7efcbc0ce930 2024-03-23T07:58:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.577+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efca40129d0 con 0x7efcbc0ce930 2024-03-23T07:58:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.577+0000 7efcb2a00640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efc9806ac00 0x7efc9806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.578+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7efca4088d70 con 0x7efcbc0ce930 2024-03-23T07:58:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.578+0000 7efcb9e00640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efc9806ac00 0x7efc9806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.578+0000 7efcb2a00640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7efc98070fa0 0x7efc980733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.578+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 --> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc98073ab0 con 0x7efc98070fa0 2024-03-23T07:58:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.578+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca408b350 con 0x7efcbc0ce930 2024-03-23T07:58:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.578+0000 7efcb9e00640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efc9806ac00 0x7efc9806d0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efcbc158170 tx=0x7efcac00b450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.579+0000 7efcba800640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7efc98070fa0 0x7efc980733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.580+0000 7efcba800640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7efc98070fa0 0x7efc980733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.582+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== osd.4 v2:172.21.15.132:6800/37441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7efc98073ab0 con 0x7efc98070fa0 2024-03-23T07:58:38.596 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:58:38.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.595+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 --> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efcbc0ccf10 con 0x7efc98070fa0 2024-03-23T07:58:38.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.596+0000 7efcb2a00640 1 -- 172.21.15.105:0/1279399583 <== osd.4 v2:172.21.15.132:6800/37441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7efcb4009450 con 0x7efc98070fa0 2024-03-23T07:58:38.597 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:38.597 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:38.597 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:38.602 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:58:38.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7efc98070fa0 msgr2=0x7efc980733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.132:6800/37441,v1:172.21.15.132:6801/37441] conn(0x7efc98070fa0 0x7efc980733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efc9806ac00 msgr2=0x7efc9806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efc9806ac00 0x7efc9806d0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efcbc158170 tx=0x7efcac00b450 comp rx=0 tx=0).stop 2024-03-23T07:58:38.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0ce930 msgr2=0x7efcbc15c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 --2- 172.21.15.105:0/1279399583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0ce930 0x7efcbc15c0b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efca400c450 tx=0x7efca400c910 comp rx=0 tx=0).stop 2024-03-23T07:58:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcba800640 1 -- 172.21.15.105:0/1279399583 reap_dead start 2024-03-23T07:58:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 shutdown_connections 2024-03-23T07:58:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.597+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 >> 172.21.15.105:0/1279399583 conn(0x7efcbc06a0b0 msgr2=0x7efcbc0ad730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.598+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 shutdown_connections 2024-03-23T07:58:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.598+0000 7efcc1200640 1 -- 172.21.15.105:0/1279399583 wait complete. 2024-03-23T07:58:38.616 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-23T07:58:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.856+0000 7ff78a200640 1 Processor -- start 2024-03-23T07:58:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.856+0000 7ff78a200640 1 -- start start 2024-03-23T07:58:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff78a200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7840ce930 0x7ff7840ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff78a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7840cb660 0x7ff7840cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff78a200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff784070170 0x7ff784070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff78a200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78415bae0 con 0x7ff7840ce930 2024-03-23T07:58:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff78a200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7840cf250 con 0x7ff784070170 2024-03-23T07:58:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff78a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78415ae60 con 0x7ff7840cb660 2024-03-23T07:58:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff783400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7840ce930 0x7ff7840ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff783400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7840ce930 0x7ff7840ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42800/0 (socket says 172.21.15.105:42800) 2024-03-23T07:58:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff783400640 1 -- 172.21.15.105:0/2692809133 learned_addr learned my addr 172.21.15.105:0/2692809133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff783400640 1 -- 172.21.15.105:0/2692809133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7840cb660 msgr2=0x7ff7840cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff783e00640 1 --2- 172.21.15.105:0/2692809133 >> v2:172.21.15.132:3300/0 conn(0x7ff784070170 0x7ff784070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.858+0000 7ff77be00640 1 --2- 172.21.15.105:0/2692809133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7840cb660 0x7ff7840cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.859+0000 7ff783400640 1 --2- 172.21.15.105:0/2692809133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7840cb660 0x7ff7840cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.859+0000 7ff783400640 1 -- 172.21.15.105:0/2692809133 >> v2:172.21.15.132:3300/0 conn(0x7ff784070170 msgr2=0x7ff784070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.859+0000 7ff783400640 1 --2- 172.21.15.105:0/2692809133 >> v2:172.21.15.132:3300/0 conn(0x7ff784070170 0x7ff784070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.859+0000 7ff783400640 1 -- 172.21.15.105:0/2692809133 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78415b140 con 0x7ff7840ce930 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.859+0000 7ff783e00640 1 --2- 172.21.15.105:0/2692809133 >> v2:172.21.15.132:3300/0 conn(0x7ff784070170 0x7ff784070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.860+0000 7ff783400640 1 --2- 172.21.15.105:0/2692809133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840ced10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff7840b3e90 tx=0x7ff77402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9fca84bed41184 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.860+0000 7ff782a00640 1 -- 172.21.15.105:0/2692809133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774031870 con 0x7ff7840ce930 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.860+0000 7ff782a00640 1 -- 172.21.15.105:0/2692809133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7740319d0 con 0x7ff7840ce930 2024-03-23T07:58:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.860+0000 7ff782a00640 1 -- 172.21.15.105:0/2692809133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774031d60 con 0x7ff7840ce930 2024-03-23T07:58:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.860+0000 7ff78a200640 1 -- 172.21.15.105:0/2692809133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 msgr2=0x7ff7840ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.860+0000 7ff78a200640 1 --2- 172.21.15.105:0/2692809133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840ced10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff7840b3e90 tx=0x7ff77402b910 comp rx=0 tx=0).stop 2024-03-23T07:58:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.861+0000 7ff78a200640 1 -- 172.21.15.105:0/2692809133 shutdown_connections 2024-03-23T07:58:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.861+0000 7ff78a200640 1 --2- 172.21.15.105:0/2692809133 >> v2:172.21.15.132:3300/0 conn(0x7ff784070170 0x7ff784070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.861+0000 7ff78a200640 1 --2- 172.21.15.105:0/2692809133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7840cb660 0x7ff7840cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.861+0000 7ff78a200640 1 --2- 172.21.15.105:0/2692809133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840ced10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.861+0000 7ff78a200640 1 -- 172.21.15.105:0/2692809133 >> 172.21.15.105:0/2692809133 conn(0x7ff78406a0b0 msgr2=0x7ff78406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.861+0000 7ff78a200640 1 -- 172.21.15.105:0/2692809133 shutdown_connections 2024-03-23T07:58:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.861+0000 7ff78a200640 1 -- 172.21.15.105:0/2692809133 wait complete. 2024-03-23T07:58:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.862+0000 7ff78a200640 1 Processor -- start 2024-03-23T07:58:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.863+0000 7ff78a200640 1 -- start start 2024-03-23T07:58:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff78a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff784070170 0x7ff78414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff78a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7840cb660 0x7ff7840ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff78a200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff78a200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840b3a30 con 0x7ff7840ce930 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff78a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840b3500 con 0x7ff7840cb660 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff78a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840b3750 con 0x7ff784070170 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42814/0 (socket says 172.21.15.105:42814) 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 -- 172.21.15.105:0/146816698 learned_addr learned my addr 172.21.15.105:0/146816698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783400640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff784070170 0x7ff78414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff77be00640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7840cb660 0x7ff7840ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 -- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff784070170 msgr2=0x7ff78414b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff784070170 0x7ff78414b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 -- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7840cb660 msgr2=0x7ff7840ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7840cb660 0x7ff7840ad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff783e00640 1 -- 172.21.15.105:0/146816698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740094d0 con 0x7ff7840ce930 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.864+0000 7ff77be00640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7840cb660 0x7ff7840ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.865+0000 7ff783e00640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840b26e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00cd10 tx=0x7ff77c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.865+0000 7ff77b400640 1 -- 172.21.15.105:0/146816698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00f680 con 0x7ff7840ce930 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.865+0000 7ff77b400640 1 -- 172.21.15.105:0/146816698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00f800 con 0x7ff7840ce930 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.865+0000 7ff77b400640 1 -- 172.21.15.105:0/146816698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00fbb0 con 0x7ff7840ce930 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.865+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff784155280 con 0x7ff7840ce930 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.865+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7840aec10 con 0x7ff7840ce930 2024-03-23T07:58:38.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.866+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7840ae2f0 con 0x7ff7840ce930 2024-03-23T07:58:38.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.867+0000 7ff77b400640 1 -- 172.21.15.105:0/146816698 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff77c00b040 con 0x7ff7840ce930 2024-03-23T07:58:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.869+0000 7ff77b400640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff76006ac30 0x7ff76006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.869+0000 7ff77b400640 1 -- 172.21.15.105:0/146816698 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ff77c089b30 con 0x7ff7840ce930 2024-03-23T07:58:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.869+0000 7ff783400640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff76006ac30 0x7ff76006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.870+0000 7ff783400640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff76006ac30 0x7ff76006d0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff774036000 tx=0x7ff77403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:38.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:38.873+0000 7ff77b400640 1 -- 172.21.15.105:0/146816698 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff7840ae2f0 con 0x7ff7840ce930 2024-03-23T07:58:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.068+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff784155100 con 0x7ff76006ac30 2024-03-23T07:58:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.069+0000 7ff77b400640 1 -- 172.21.15.105:0/146816698 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff774031cc0 con 0x7ff76006ac30 2024-03-23T07:58:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 4 to deep-scrub 2024-03-23T07:58:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff76006ac30 msgr2=0x7ff76006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff76006ac30 0x7ff76006d0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff774036000 tx=0x7ff77403c040 comp rx=0 tx=0).stop 2024-03-23T07:58:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 msgr2=0x7ff7840b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840b26e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00cd10 tx=0x7ff77c00a650 comp rx=0 tx=0).stop 2024-03-23T07:58:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 shutdown_connections 2024-03-23T07:58:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7840ce930 0x7ff7840b26e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7840cb660 0x7ff7840ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7ff76006ac30 0x7ff76006d0f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 --2- 172.21.15.105:0/146816698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff784070170 0x7ff78414b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.071+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 >> 172.21.15.105:0/146816698 conn(0x7ff78406a0b0 msgr2=0x7ff78415bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:39.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.072+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 shutdown_connections 2024-03-23T07:58:39.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.072+0000 7ff78a200640 1 -- 172.21.15.105:0/146816698 wait complete. 2024-03-23T07:58:39.091 INFO:tasks.ceph:Scrubbing osd.5 2024-03-23T07:58:39.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:39.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.307+0000 7f6ff9200640 1 Processor -- start 2024-03-23T07:58:39.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.307+0000 7f6ff9200640 1 -- start start 2024-03-23T07:58:39.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.309+0000 7f6ff9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40cd740 0x7f6ff40b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.309+0000 7f6ff9200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6ff40b5aa0 0x7f6ff40b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.309+0000 7f6ff9200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40d20c0 con 0x7f6ff40d0d40 2024-03-23T07:58:39.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.309+0000 7f6ff9200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff40d14b0 con 0x7f6ff40b5aa0 2024-03-23T07:58:39.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.309+0000 7f6ff9200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40d1790 con 0x7f6ff40cd740 2024-03-23T07:58:39.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.310+0000 7f6ff1400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6ff40d0d40 0x7f6ff40cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.105:58164/0 (socket says 172.21.15.105:58164) 2024-03-23T07:58:39.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.310+0000 7f6ff1400640 1 -- 172.21.15.105:0/2171690758 learned_addr learned my addr 172.21.15.105:0/2171690758 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:39.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.310+0000 7f6ff1e00640 1 --2- 172.21.15.105:0/2171690758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40cd740 0x7f6ff40b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.310+0000 7f6ff2800640 1 --2- 172.21.15.105:0/2171690758 >> v2:172.21.15.132:3300/0 conn(0x7f6ff40b5aa0 0x7f6ff40b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.311+0000 7f6ff1e00640 1 -- 172.21.15.105:0/2171690758 >> v2:172.21.15.132:3300/0 conn(0x7f6ff40b5aa0 msgr2=0x7f6ff40b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:39.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.311+0000 7f6ff1e00640 1 --2- 172.21.15.105:0/2171690758 >> v2:172.21.15.132:3300/0 conn(0x7f6ff40b5aa0 0x7f6ff40b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.311+0000 7f6ff1e00640 1 -- 172.21.15.105:0/2171690758 >> v1:172.21.15.132:6789/0 conn(0x7f6ff40d0d40 legacy=0x7f6ff40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.311+0000 7f6ff1e00640 1 -- 172.21.15.105:0/2171690758 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40d1a70 con 0x7f6ff40cd740 2024-03-23T07:58:39.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.311+0000 7f6ff2800640 1 --2- 172.21.15.105:0/2171690758 >> v2:172.21.15.132:3300/0 conn(0x7f6ff40b5aa0 0x7f6ff40b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T07:58:39.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.312+0000 7f6ff1e00640 1 --2- 172.21.15.105:0/2171690758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40cd740 0x7f6ff40b84b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40097a0 tx=0x7f6fe402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acbb3cf9be560afd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:39.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.313+0000 7f6ff0a00640 1 -- 172.21.15.105:0/2171690758 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4031830 con 0x7f6ff40cd740 2024-03-23T07:58:39.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.313+0000 7f6ff0a00640 1 -- 172.21.15.105:0/2171690758 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe4031990 con 0x7f6ff40cd740 2024-03-23T07:58:39.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.313+0000 7f6ff0a00640 1 -- 172.21.15.105:0/2171690758 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4031d20 con 0x7f6ff40cd740 2024-03-23T07:58:39.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 -- 172.21.15.105:0/2171690758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40cd740 msgr2=0x7f6ff40b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 --2- 172.21.15.105:0/2171690758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40cd740 0x7f6ff40b84b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40097a0 tx=0x7f6fe402b8d0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 -- 172.21.15.105:0/2171690758 shutdown_connections 2024-03-23T07:58:39.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 --2- 172.21.15.105:0/2171690758 >> v2:172.21.15.132:3300/0 conn(0x7f6ff40b5aa0 0x7f6ff40b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 --2- 172.21.15.105:0/2171690758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40cd740 0x7f6ff40b84b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 -- 172.21.15.105:0/2171690758 >> 172.21.15.105:0/2171690758 conn(0x7f6ff40cf5d0 msgr2=0x7f6ff40cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:39.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 -- 172.21.15.105:0/2171690758 shutdown_connections 2024-03-23T07:58:39.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.314+0000 7f6ff9200640 1 -- 172.21.15.105:0/2171690758 wait complete. 2024-03-23T07:58:39.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.315+0000 7f6ff9200640 1 Processor -- start 2024-03-23T07:58:39.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.316+0000 7f6ff9200640 1 -- start start 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.316+0000 7f6ff9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40b5aa0 0x7f6ff41f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.316+0000 7f6ff9200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40cd740 0x7f6ff415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d0d40 0x7f6ff415e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff9200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40aa620 con 0x7f6ff40d0d40 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff9200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40a99e0 con 0x7f6ff40cd740 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff9200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40a9c80 con 0x7f6ff40b5aa0 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d0d40 0x7f6ff415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d0d40 0x7f6ff415e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42818/0 (socket says 172.21.15.105:42818) 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 -- 172.21.15.105:0/752720318 learned_addr learned my addr 172.21.15.105:0/752720318 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff1e00640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40b5aa0 0x7f6ff41f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff1400640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40cd740 0x7f6ff415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 -- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40b5aa0 msgr2=0x7f6ff41f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40b5aa0 0x7f6ff41f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 -- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40cd740 msgr2=0x7f6ff415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40cd740 0x7f6ff415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff2800640 1 -- 172.21.15.105:0/752720318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec009400 con 0x7f6ff40d0d40 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff1e00640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ff40b5aa0 0x7f6ff41f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.317+0000 7f6ff1400640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40cd740 0x7f6ff415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:39.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.318+0000 7f6ff2800640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d0d40 0x7f6ff415e900 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00e930 tx=0x7f6fec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:39.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.318+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec009700 con 0x7f6ff40d0d40 2024-03-23T07:58:39.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.318+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec00ca00 con 0x7f6ff40d0d40 2024-03-23T07:58:39.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.318+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00cd90 con 0x7f6ff40d0d40 2024-03-23T07:58:39.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.318+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe4009400 con 0x7f6ff40d0d40 2024-03-23T07:58:39.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.318+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff41f67f0 con 0x7f6ff40d0d40 2024-03-23T07:58:39.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.319+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ff4160300 con 0x7f6ff40d0d40 2024-03-23T07:58:39.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.320+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6fec013470 con 0x7f6ff40d0d40 2024-03-23T07:58:39.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.321+0000 7f6fea000640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6fd006ac00 0x7f6fd006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.321+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6fec089470 con 0x7f6ff40d0d40 2024-03-23T07:58:39.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.321+0000 7f6ff1e00640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6fd006ac00 0x7f6fd006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.321+0000 7f6fea000640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f6fd0070fa0 0x7f6fd00733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.321+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 --> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fd0073ab0 con 0x7f6fd0070fa0 2024-03-23T07:58:39.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.321+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fec089820 con 0x7f6ff40d0d40 2024-03-23T07:58:39.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.322+0000 7f6ff1e00640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6fd006ac00 0x7f6fd006d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4036000 tx=0x7f6fe4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:39.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.322+0000 7f6ff1400640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f6fd0070fa0 0x7f6fd00733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.323+0000 7f6ff1400640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f6fd0070fa0 0x7f6fd00733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:39.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.325+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== osd.5 v2:172.21.15.132:6817/37443 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6fd0073ab0 con 0x7f6fd0070fa0 2024-03-23T07:58:39.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.339+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 --> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6ff40d11a0 con 0x7f6fd0070fa0 2024-03-23T07:58:39.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6fea000640 1 -- 172.21.15.105:0/752720318 <== osd.5 v2:172.21.15.132:6817/37443 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6fdc0070e0 con 0x7f6fd0070fa0 2024-03-23T07:58:39.340 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:39.341 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:39.341 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f6fd0070fa0 msgr2=0x7f6fd00733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6ff9200640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.132:6817/37443,v1:172.21.15.132:6819/37443] conn(0x7f6fd0070fa0 0x7f6fd00733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6fd006ac00 msgr2=0x7f6fd006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6ff9200640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f6fd006ac00 0x7f6fd006d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4036000 tx=0x7f6fe4008030 comp rx=0 tx=0).stop 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d0d40 msgr2=0x7f6ff415e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6ff9200640 1 --2- 172.21.15.105:0/752720318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d0d40 0x7f6ff415e900 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00e930 tx=0x7f6fec00edf0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.340+0000 7f6ff2800640 1 -- 172.21.15.105:0/752720318 reap_dead start 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.341+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 shutdown_connections 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.341+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 >> 172.21.15.105:0/752720318 conn(0x7f6ff40cf5d0 msgr2=0x7f6ff40b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.341+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 shutdown_connections 2024-03-23T07:58:39.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.341+0000 7f6ff9200640 1 -- 172.21.15.105:0/752720318 wait complete. 2024-03-23T07:58:39.361 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-23T07:58:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.571+0000 7f7b69200640 1 Processor -- start 2024-03-23T07:58:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.571+0000 7f7b69200640 1 -- start start 2024-03-23T07:58:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.573+0000 7f7b69200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b640ce930 0x7f7b640ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.573+0000 7f7b69200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b64070f90 con 0x7f7b6406d2c0 2024-03-23T07:58:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.573+0000 7f7b69200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b640cf250 con 0x7f7b640cb660 2024-03-23T07:58:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.573+0000 7f7b69200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6415bae0 con 0x7f7b640ce930 2024-03-23T07:58:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.573+0000 7f7b62800640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7b6406d2c0 0x7f7b64070880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59522/0 (socket says 172.21.15.105:59522) 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.573+0000 7f7b62800640 1 -- 172.21.15.105:0/1596431347 learned_addr learned my addr 172.21.15.105:0/1596431347 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b61e00640 1 --2- 172.21.15.105:0/1596431347 >> v2:172.21.15.132:3300/0 conn(0x7f7b640ce930 0x7f7b640ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095505449 0 0) 0x7f7b64070f90 con 0x7f7b6406d2c0 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b48003520 con 0x7f7b6406d2c0 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848932214 0 0) 0x7f7b48003520 con 0x7f7b6406d2c0 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 >> v2:172.21.15.132:3300/0 conn(0x7f7b640ce930 msgr2=0x7f7b640ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 --2- 172.21.15.105:0/1596431347 >> v2:172.21.15.132:3300/0 conn(0x7f7b640ce930 0x7f7b640ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 >> v1:172.21.15.132:6789/0 conn(0x7f7b640cb660 legacy=0x7f7b64070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6415ae60 con 0x7f7b6406d2c0 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f7b5c0019f0 con 0x7f7b6406d2c0 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 >> v1:172.21.15.105:6789/0 conn(0x7f7b6406d2c0 legacy=0x7f7b64070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b480047c0 0x7f7b48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b48005180 0x7f7b4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6415bae0 con 0x7f7b48005180 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4800c080 con 0x7f7b480039c0 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.574+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4800c240 con 0x7f7b480047c0 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.575+0000 7f7b62800640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b48005180 0x7f7b4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.575+0000 7f7b62800640 1 -- 172.21.15.105:0/1596431347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b480047c0 msgr2=0x7f7b48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.575+0000 7f7b62800640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b480047c0 0x7f7b48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.575+0000 7f7b62800640 1 -- 172.21.15.105:0/1596431347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b480039c0 msgr2=0x7f7b48003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.575+0000 7f7b61e00640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.575+0000 7f7b62800640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.575+0000 7f7b62800640 1 -- 172.21.15.105:0/1596431347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48003520 con 0x7f7b48005180 2024-03-23T07:58:39.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.576+0000 7f7b62800640 1 -- 172.21.15.105:0/1596431347 reap_dead start 2024-03-23T07:58:39.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.576+0000 7f7b62800640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b48005180 0x7f7b4800b980 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00a760 tx=0x7f7b480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ec24b0250950f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:39.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c034630 con 0x7f7b48005180 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c02f070 con 0x7f7b48005180 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b60a00640 1 -- 172.21.15.105:0/1596431347 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c034a10 con 0x7f7b48005180 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b69200640 1 -- 172.21.15.105:0/1596431347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b48005180 msgr2=0x7f7b4800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b69200640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b48005180 0x7f7b4800b980 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00a760 tx=0x7f7b480057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b69200640 1 -- 172.21.15.105:0/1596431347 shutdown_connections 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b69200640 1 --2- 172.21.15.105:0/1596431347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b48005180 0x7f7b4800b980 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b69200640 1 -- 172.21.15.105:0/1596431347 >> 172.21.15.105:0/1596431347 conn(0x7f7b6406a0b0 msgr2=0x7f7b6406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b69200640 1 -- 172.21.15.105:0/1596431347 shutdown_connections 2024-03-23T07:58:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.577+0000 7f7b69200640 1 -- 172.21.15.105:0/1596431347 wait complete. 2024-03-23T07:58:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.579+0000 7f7b69200640 1 Processor -- start 2024-03-23T07:58:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.579+0000 7f7b69200640 1 -- start start 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b69200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b48005180 0x7f7b6414f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b69200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b64156980 0x7f7b64155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b69200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64150160 0x7f7b641505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b69200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641f0010 con 0x7f7b64150160 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b69200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64150b20 con 0x7f7b48005180 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b69200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b640afe00 con 0x7f7b64156980 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b62800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64150160 0x7f7b641505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b62800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64150160 0x7f7b641505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42838/0 (socket says 172.21.15.105:42838) 2024-03-23T07:58:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.580+0000 7f7b62800640 1 -- 172.21.15.105:0/3723116845 learned_addr learned my addr 172.21.15.105:0/3723116845 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b61400640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b64156980 0x7f7b64155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b61e00640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b48005180 0x7f7b6414f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b62800640 1 -- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b64156980 msgr2=0x7f7b64155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b62800640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b64156980 0x7f7b64155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b62800640 1 -- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b48005180 msgr2=0x7f7b6414f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b62800640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b48005180 0x7f7b6414f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b62800640 1 -- 172.21.15.105:0/3723116845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b641558b0 con 0x7f7b64150160 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b61400640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b64156980 0x7f7b64155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.581+0000 7f7b62800640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64150160 0x7f7b641505e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c002230 tx=0x7f7b5c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.582+0000 7f7b5a000640 1 -- 172.21.15.105:0/3723116845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0349b0 con 0x7f7b64150160 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.582+0000 7f7b5a000640 1 -- 172.21.15.105:0/3723116845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c034b10 con 0x7f7b64150160 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.582+0000 7f7b5a000640 1 -- 172.21.15.105:0/3723116845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c034ea0 con 0x7f7b64150160 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.582+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b64155b50 con 0x7f7b64150160 2024-03-23T07:58:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.582+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b64156110 con 0x7f7b64150160 2024-03-23T07:58:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.583+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b640b1270 con 0x7f7b64150160 2024-03-23T07:58:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.584+0000 7f7b5a000640 1 -- 172.21.15.105:0/3723116845 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7b5c031030 con 0x7f7b64150160 2024-03-23T07:58:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.585+0000 7f7b5a000640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7b4006ac30 0x7f7b4006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.585+0000 7f7b5a000640 1 -- 172.21.15.105:0/3723116845 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f7b5c0aab40 con 0x7f7b64150160 2024-03-23T07:58:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.586+0000 7f7b61e00640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7b4006ac30 0x7f7b4006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.586+0000 7f7b61e00640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7b4006ac30 0x7f7b4006d0f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009a10 tx=0x7f7b54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.590+0000 7f7b5a000640 1 -- 172.21.15.105:0/3723116845 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7b5c075e80 con 0x7f7b64150160 2024-03-23T07:58:39.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.784+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b6415b700 con 0x7f7b4006ac30 2024-03-23T07:58:39.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.785+0000 7f7b5a000640 1 -- 172.21.15.105:0/3723116845 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7b54004020 con 0x7f7b4006ac30 2024-03-23T07:58:39.785 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 5 to deep-scrub 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7b4006ac30 msgr2=0x7f7b4006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7b4006ac30 0x7f7b4006d0f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009a10 tx=0x7f7b54008040 comp rx=0 tx=0).stop 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64150160 msgr2=0x7f7b641505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64150160 0x7f7b641505e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c002230 tx=0x7f7b5c001cb0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 shutdown_connections 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64150160 0x7f7b641505e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f7b4006ac30 0x7f7b4006d0f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b64156980 0x7f7b64155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 --2- 172.21.15.105:0/3723116845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b48005180 0x7f7b6414f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:39.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 >> 172.21.15.105:0/3723116845 conn(0x7f7b6406a0b0 msgr2=0x7f7b64154080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:39.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 shutdown_connections 2024-03-23T07:58:39.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:39.787+0000 7f7b69200640 1 -- 172.21.15.105:0/3723116845 wait complete. 2024-03-23T07:58:39.806 INFO:tasks.ceph:Scrubbing osd.6 2024-03-23T07:58:39.806 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:40.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.014+0000 7f22e5e00640 1 Processor -- start 2024-03-23T07:58:40.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.014+0000 7f22e5e00640 1 -- start start 2024-03-23T07:58:40.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.015+0000 7f22e5e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22e00ce930 0x7f22e00add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.015+0000 7f22e5e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f22e006ce90 0x7f22e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.015+0000 7f22e5e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00cdba0 con 0x7f22e00cd3e0 2024-03-23T07:58:40.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22e5e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e006c760 con 0x7f22e00ce930 2024-03-23T07:58:40.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22e5e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00b42d0 con 0x7f22e006ce90 2024-03-23T07:58:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22dea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22e00ce930 0x7f22e00add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22dea00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22e00ce930 0x7f22e00add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42840/0 (socket says 172.21.15.105:42840) 2024-03-23T07:58:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22dea00640 1 -- 172.21.15.105:0/3405574046 learned_addr learned my addr 172.21.15.105:0/3405574046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22de000640 1 --1- 172.21.15.105:0/3405574046 >> v1:172.21.15.105:6789/0 conn(0x7f22e00cd3e0 0x7f22e00ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59534/0 (socket says 172.21.15.105:59534) 2024-03-23T07:58:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22df400640 1 --2- 172.21.15.105:0/3405574046 >> v2:172.21.15.132:3300/0 conn(0x7f22e006ce90 0x7f22e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22dea00640 1 -- 172.21.15.105:0/3405574046 >> v2:172.21.15.132:3300/0 conn(0x7f22e006ce90 msgr2=0x7f22e006d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22dea00640 1 --2- 172.21.15.105:0/3405574046 >> v2:172.21.15.132:3300/0 conn(0x7f22e006ce90 0x7f22e006d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22dea00640 1 -- 172.21.15.105:0/3405574046 >> v1:172.21.15.105:6789/0 conn(0x7f22e00cd3e0 legacy=0x7f22e00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.016+0000 7f22dea00640 1 -- 172.21.15.105:0/3405574046 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00cde80 con 0x7f22e00ce930 2024-03-23T07:58:40.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.017+0000 7f22dea00640 1 --2- 172.21.15.105:0/3405574046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e00add30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22e006c4b0 tx=0x7f22d002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312d2d7f55581b19 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22dd600640 1 -- 172.21.15.105:0/3405574046 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0030870 con 0x7f22e00ce930 2024-03-23T07:58:40.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22dd600640 1 -- 172.21.15.105:0/3405574046 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d00309d0 con 0x7f22e00ce930 2024-03-23T07:58:40.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22dd600640 1 -- 172.21.15.105:0/3405574046 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0030d60 con 0x7f22e00ce930 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22e5e00640 1 -- 172.21.15.105:0/3405574046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 msgr2=0x7f22e00add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22e5e00640 1 --2- 172.21.15.105:0/3405574046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e00add30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22e006c4b0 tx=0x7f22d002b9b0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22e5e00640 1 -- 172.21.15.105:0/3405574046 shutdown_connections 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22e5e00640 1 --2- 172.21.15.105:0/3405574046 >> v2:172.21.15.132:3300/0 conn(0x7f22e006ce90 0x7f22e006d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.018+0000 7f22e5e00640 1 --2- 172.21.15.105:0/3405574046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e00add30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.019+0000 7f22e5e00640 1 -- 172.21.15.105:0/3405574046 >> 172.21.15.105:0/3405574046 conn(0x7f22e006a0b0 msgr2=0x7f22e006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.019+0000 7f22e5e00640 1 -- 172.21.15.105:0/3405574046 shutdown_connections 2024-03-23T07:58:40.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.019+0000 7f22e5e00640 1 -- 172.21.15.105:0/3405574046 wait complete. 2024-03-23T07:58:40.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.020+0000 7f22e5e00640 1 Processor -- start 2024-03-23T07:58:40.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.021+0000 7f22e5e00640 1 -- start start 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.021+0000 7f22e5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22e006ce90 0x7f22e015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.021+0000 7f22e5e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22e00cd3e0 0x7f22e013cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.021+0000 7f22e5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e0159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.021+0000 7f22e5e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015b710 con 0x7f22e00ce930 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.021+0000 7f22e5e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e013d940 con 0x7f22e006ce90 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.021+0000 7f22e5e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e013dc20 con 0x7f22e00cd3e0 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e0159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e0159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42848/0 (socket says 172.21.15.105:42848) 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 -- 172.21.15.105:0/2488793435 learned_addr learned my addr 172.21.15.105:0/2488793435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22dea00640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22e006ce90 0x7f22e015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22de000640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22e00cd3e0 0x7f22e013cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 -- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22e00cd3e0 msgr2=0x7f22e013cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22e00cd3e0 0x7f22e013cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 -- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22e006ce90 msgr2=0x7f22e015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22e006ce90 0x7f22e015b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22df400640 1 -- 172.21.15.105:0/2488793435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00094d0 con 0x7f22e00ce930 2024-03-23T07:58:40.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22dea00640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22e006ce90 0x7f22e015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:40.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.022+0000 7f22de000640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22e00cd3e0 0x7f22e013cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:40.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.023+0000 7f22df400640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e0159170 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f22d8007e40 tx=0x7f22d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.023+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8014620 con 0x7f22e00ce930 2024-03-23T07:58:40.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.023+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d800ee90 con 0x7f22e00ce930 2024-03-23T07:58:40.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.023+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e013df00 con 0x7f22e00ce930 2024-03-23T07:58:40.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.023+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80149e0 con 0x7f22e00ce930 2024-03-23T07:58:40.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.023+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e013e4c0 con 0x7f22e00ce930 2024-03-23T07:58:40.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.024+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22e015df90 con 0x7f22e00ce930 2024-03-23T07:58:40.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.025+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f22d8014b40 con 0x7f22e00ce930 2024-03-23T07:58:40.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.026+0000 7f22d7400640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22c006ac00 0x7f22c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.026+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f22d8089be0 con 0x7f22e00ce930 2024-03-23T07:58:40.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.026+0000 7f22dea00640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22c006ac00 0x7f22c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.026+0000 7f22d7400640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7f22c0070fa0 0x7f22c00733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.026+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 --> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22c0073ab0 con 0x7f22c0070fa0 2024-03-23T07:58:40.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.026+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d808e2a0 con 0x7f22e00ce930 2024-03-23T07:58:40.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.027+0000 7f22dea00640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22c006ac00 0x7f22c006d0c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f22d0036000 tx=0x7f22d003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.027+0000 7f22de000640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7f22c0070fa0 0x7f22c00733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.028+0000 7f22de000640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7f22c0070fa0 0x7f22c00733e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.030+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== osd.6 v2:172.21.15.132:6808/37442 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f22c0073ab0 con 0x7f22c0070fa0 2024-03-23T07:58:40.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.043+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 --> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f22e006c970 con 0x7f22c0070fa0 2024-03-23T07:58:40.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22d7400640 1 -- 172.21.15.105:0/2488793435 <== osd.6 v2:172.21.15.132:6808/37442 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f22cc001820 con 0x7f22c0070fa0 2024-03-23T07:58:40.045 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:40.045 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:40.045 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:40.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7f22c0070fa0 msgr2=0x7f22c00733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22e5e00640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.132:6808/37442,v1:172.21.15.132:6809/37442] conn(0x7f22c0070fa0 0x7f22c00733e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22c006ac00 msgr2=0x7f22c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22e5e00640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f22c006ac00 0x7f22c006d0c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f22d0036000 tx=0x7f22d003b040 comp rx=0 tx=0).stop 2024-03-23T07:58:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 msgr2=0x7f22e0159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22e5e00640 1 --2- 172.21.15.105:0/2488793435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22e00ce930 0x7f22e0159170 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f22d8007e40 tx=0x7f22d800e4b0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.045+0000 7f22df400640 1 -- 172.21.15.105:0/2488793435 reap_dead start 2024-03-23T07:58:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.046+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 shutdown_connections 2024-03-23T07:58:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.046+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 >> 172.21.15.105:0/2488793435 conn(0x7f22e006a0b0 msgr2=0x7f22e00b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.046+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 shutdown_connections 2024-03-23T07:58:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.046+0000 7f22e5e00640 1 -- 172.21.15.105:0/2488793435 wait complete. 2024-03-23T07:58:40.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-23T07:58:40.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.278+0000 7fc5b6000640 1 Processor -- start 2024-03-23T07:58:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.278+0000 7fc5b6000640 1 -- start start 2024-03-23T07:58:40.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.279+0000 7fc5b6000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc5b0159690 0x7fc5b00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.279+0000 7fc5b6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 0x7fc5b00cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.279+0000 7fc5b6000640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b015e010 con 0x7fc5b00cc330 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.279+0000 7fc5b6000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b00ce1f0 con 0x7fc5b0159690 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.279+0000 7fc5b6000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b00cc7f0 con 0x7fc5b00cd890 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.280+0000 7fc5a6a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc5b00cc330 0x7fc5b00b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59538/0 (socket says 172.21.15.105:59538) 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.280+0000 7fc5a6a00640 1 -- 172.21.15.105:0/2070959702 learned_addr learned my addr 172.21.15.105:0/2070959702 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.280+0000 7fc5afe00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 0x7fc5b00cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.280+0000 7fc5af400640 1 --2- 172.21.15.105:0/2070959702 >> v2:172.21.15.132:3300/0 conn(0x7fc5b0159690 0x7fc5b00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.280+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897607379 0 0) 0x7fc5b015e010 con 0x7fc5b00cc330 2024-03-23T07:58:40.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.280+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc594003520 con 0x7fc5b00cc330 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4273507442 0 0) 0x7fc594003520 con 0x7fc5b00cc330 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 msgr2=0x7fc5b00cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 0x7fc5b00cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 >> v2:172.21.15.132:3300/0 conn(0x7fc5b0159690 msgr2=0x7fc5b00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 --2- 172.21.15.105:0/2070959702 >> v2:172.21.15.132:3300/0 conn(0x7fc5b0159690 0x7fc5b00ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b00ccad0 con 0x7fc5b00cc330 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fc598001670 con 0x7fc5b00cc330 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 >> v1:172.21.15.105:6789/0 conn(0x7fc5b00cc330 legacy=0x7fc5b00b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 0x7fc594003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc594003ef0 0x7fc5940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc594005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b00cc7f0 con 0x7fc594004d00 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59400db90 con 0x7fc594003ef0 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59400dcd0 con 0x7fc5b00cd890 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5afe00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc594005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5af400640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 0x7fc594003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.281+0000 7fc5a6a00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc594003ef0 0x7fc5940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.282+0000 7fc5afe00640 1 -- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 msgr2=0x7fc594003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.282+0000 7fc5afe00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b00cd890 0x7fc594003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.282+0000 7fc5afe00640 1 -- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc594003ef0 msgr2=0x7fc5940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.282+0000 7fc5afe00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc594003ef0 0x7fc5940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.282+0000 7fc5afe00640 1 -- 172.21.15.105:0/2070959702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594003520 con 0x7fc594004d00 2024-03-23T07:58:40.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.283+0000 7fc5afe00640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc594005180 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009ac0 tx=0x7fc5940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5591137d78ea3925 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.283+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a802c6b0 con 0x7fc594004d00 2024-03-23T07:58:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.283+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a802c810 con 0x7fc594004d00 2024-03-23T07:58:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.283+0000 7fc5aea00640 1 -- 172.21.15.105:0/2070959702 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a802cba0 con 0x7fc594004d00 2024-03-23T07:58:40.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.284+0000 7fc5b6000640 1 -- 172.21.15.105:0/2070959702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 msgr2=0x7fc594005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.284+0000 7fc5b6000640 1 --2- 172.21.15.105:0/2070959702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc594005180 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009ac0 tx=0x7fc5940057f0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.284+0000 7fc5afe00640 1 -- 172.21.15.105:0/2070959702 reap_dead start 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.284+0000 7fc5b6000640 1 -- 172.21.15.105:0/2070959702 shutdown_connections 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.284+0000 7fc5b6000640 1 -- 172.21.15.105:0/2070959702 >> 172.21.15.105:0/2070959702 conn(0x7fc5b006a0b0 msgr2=0x7fc5b006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.284+0000 7fc5b6000640 1 -- 172.21.15.105:0/2070959702 shutdown_connections 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.284+0000 7fc5b6000640 1 -- 172.21.15.105:0/2070959702 wait complete. 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.286+0000 7fc5b6000640 1 Processor -- start 2024-03-23T07:58:40.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.286+0000 7fc5b6000640 1 -- start start 2024-03-23T07:58:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5b6000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc5b013f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5b6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b014ea20 0x7fc5b00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5b6000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5b014eea0 0x7fc5b00b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5b6000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b01eeb20 con 0x7fc594004d00 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5b6000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b013f6a0 con 0x7fc5b014eea0 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5b6000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b01f11a0 con 0x7fc5b014ea20 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc5b013f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc5b013f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42868/0 (socket says 172.21.15.105:42868) 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 -- 172.21.15.105:0/2896972774 learned_addr learned my addr 172.21.15.105:0/2896972774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5a6a00640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b014ea20 0x7fc5b00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 -- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b014ea20 msgr2=0x7fc5b00b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b014ea20 0x7fc5b00b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5afe00640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5b014eea0 0x7fc5b00b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 -- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5b014eea0 msgr2=0x7fc5b00b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5b014eea0 0x7fc5b00b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.287+0000 7fc5af400640 1 -- 172.21.15.105:0/2896972774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b01ee560 con 0x7fc594004d00 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.288+0000 7fc5af400640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc5b013f160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0011e30 tx=0x7fc5a000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.288+0000 7fc5a7e00640 1 -- 172.21.15.105:0/2896972774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e040 con 0x7fc594004d00 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.288+0000 7fc5a7e00640 1 -- 172.21.15.105:0/2896972774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000f040 con 0x7fc594004d00 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.288+0000 7fc5a7e00640 1 -- 172.21.15.105:0/2896972774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0009c00 con 0x7fc594004d00 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.288+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b01ee840 con 0x7fc594004d00 2024-03-23T07:58:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.288+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b00aff50 con 0x7fc594004d00 2024-03-23T07:58:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.289+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b0073390 con 0x7fc594004d00 2024-03-23T07:58:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.290+0000 7fc5a7e00640 1 -- 172.21.15.105:0/2896972774 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc5a0015850 con 0x7fc594004d00 2024-03-23T07:58:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.291+0000 7fc5a7e00640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc58c06ac30 0x7fc58c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.291+0000 7fc5a7e00640 1 -- 172.21.15.105:0/2896972774 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc5a008a070 con 0x7fc594004d00 2024-03-23T07:58:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.291+0000 7fc5a6a00640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc58c06ac30 0x7fc58c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.292+0000 7fc5a6a00640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc58c06ac30 0x7fc58c06d0f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00afa80 tx=0x7fc598001d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.295+0000 7fc5a7e00640 1 -- 172.21.15.105:0/2896972774 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc5a00bd580 con 0x7fc594004d00 2024-03-23T07:58:40.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.491+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5b00afaf0 con 0x7fc58c06ac30 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.492+0000 7fc5a7e00640 1 -- 172.21.15.105:0/2896972774 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc598005e60 con 0x7fc58c06ac30 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 6 to deep-scrub 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc58c06ac30 msgr2=0x7fc58c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc58c06ac30 0x7fc58c06d0f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00afa80 tx=0x7fc598001d70 comp rx=0 tx=0).stop 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 msgr2=0x7fc5b013f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc5b013f160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0011e30 tx=0x7fc5a000f580 comp rx=0 tx=0).stop 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 shutdown_connections 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5b014eea0 0x7fc5b00b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc58c06ac30 0x7fc58c06d0f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5b014ea20 0x7fc5b00b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 --2- 172.21.15.105:0/2896972774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc594004d00 0x7fc5b013f160 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 >> 172.21.15.105:0/2896972774 conn(0x7fc5b006a0b0 msgr2=0x7fc5b0153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:40.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 shutdown_connections 2024-03-23T07:58:40.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.495+0000 7fc5b6000640 1 -- 172.21.15.105:0/2896972774 wait complete. 2024-03-23T07:58:40.515 INFO:tasks.ceph:Scrubbing osd.7 2024-03-23T07:58:40.515 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-23T07:58:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.730+0000 7f642e200640 1 Processor -- start 2024-03-23T07:58:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.730+0000 7f642e200640 1 -- start start 2024-03-23T07:58:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f642e200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6428159830 0x7f64280cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f642e200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6428159c10 con 0x7f64280ca7b0 2024-03-23T07:58:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f642e200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280cd840 con 0x7f64280ce930 2024-03-23T07:58:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f642e200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642806de50 con 0x7f6428159830 2024-03-23T07:58:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f6427400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6428159830 0x7f64280cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f6426a00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f64280ca7b0 0x7f64280cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59552/0 (socket says 172.21.15.105:59552) 2024-03-23T07:58:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f6426a00640 1 -- 172.21.15.105:0/1292964728 learned_addr learned my addr 172.21.15.105:0/1292964728 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f6427400640 1 -- 172.21.15.105:0/1292964728 >> v1:172.21.15.132:6789/0 conn(0x7f64280ce930 legacy=0x7f64280b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f6427400640 1 -- 172.21.15.105:0/1292964728 >> v1:172.21.15.105:6789/0 conn(0x7f64280ca7b0 legacy=0x7f64280cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f6427400640 1 -- 172.21.15.105:0/1292964728 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280ccc30 con 0x7f6428159830 2024-03-23T07:58:40.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.732+0000 7f6426000640 1 -- 172.21.15.105:0/1292964728 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72052133 0 0) 0x7f6428159c10 con 0x7f64280ca7b0 2024-03-23T07:58:40.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.733+0000 7f6427400640 1 --2- 172.21.15.105:0/1292964728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6428159830 0x7f64280cc3c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f64180097d0 tx=0x7f641802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3612a97d2e70b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.733+0000 7f6426000640 1 -- 172.21.15.105:0/1292964728 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418030610 con 0x7f6428159830 2024-03-23T07:58:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.733+0000 7f6426000640 1 -- 172.21.15.105:0/1292964728 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6418030770 con 0x7f6428159830 2024-03-23T07:58:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.733+0000 7f6426000640 1 -- 172.21.15.105:0/1292964728 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418030b00 con 0x7f6428159830 2024-03-23T07:58:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.734+0000 7f642e200640 1 -- 172.21.15.105:0/1292964728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6428159830 msgr2=0x7f64280cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.734+0000 7f642e200640 1 --2- 172.21.15.105:0/1292964728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6428159830 0x7f64280cc3c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f64180097d0 tx=0x7f641802b960 comp rx=0 tx=0).stop 2024-03-23T07:58:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.734+0000 7f642e200640 1 -- 172.21.15.105:0/1292964728 shutdown_connections 2024-03-23T07:58:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.734+0000 7f642e200640 1 --2- 172.21.15.105:0/1292964728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6428159830 0x7f64280cc3c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.734+0000 7f642e200640 1 -- 172.21.15.105:0/1292964728 >> 172.21.15.105:0/1292964728 conn(0x7f642806a0b0 msgr2=0x7f642806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.734+0000 7f642e200640 1 -- 172.21.15.105:0/1292964728 shutdown_connections 2024-03-23T07:58:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.734+0000 7f642e200640 1 -- 172.21.15.105:0/1292964728 wait complete. 2024-03-23T07:58:40.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.735+0000 7f642e200640 1 Processor -- start 2024-03-23T07:58:40.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.736+0000 7f642e200640 1 -- start start 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64280ca7b0 0x7f642814ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64280ce930 0x7f64281542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6428159830 0x7f642813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6428150230 con 0x7f64280ce930 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64281547f0 con 0x7f64280ca7b0 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642814f5b0 con 0x7f6428159830 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64280ce930 0x7f64281542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64280ce930 0x7f64281542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42884/0 (socket says 172.21.15.105:42884) 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 -- 172.21.15.105:0/2810581487 learned_addr learned my addr 172.21.15.105:0/2810581487 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:40.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6427400640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64280ca7b0 0x7f642814ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 -- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6428159830 msgr2=0x7f642813e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6428159830 0x7f642813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 -- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64280ca7b0 msgr2=0x7f642814ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64280ca7b0 0x7f642814ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 -- 172.21.15.105:0/2810581487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418009400 con 0x7f64280ce930 2024-03-23T07:58:40.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f6426a00640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64280ce930 0x7f64281542b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f641400c4d0 tx=0x7f641400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64140074e0 con 0x7f64280ce930 2024-03-23T07:58:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6414007660 con 0x7f64280ce930 2024-03-23T07:58:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642814f890 con 0x7f64280ce930 2024-03-23T07:58:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414004a20 con 0x7f64280ce930 2024-03-23T07:58:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.738+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642814fe50 con 0x7f64280ce930 2024-03-23T07:58:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.739+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64280cd4d0 con 0x7f64280ce930 2024-03-23T07:58:40.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.740+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6414004b80 con 0x7f64280ce930 2024-03-23T07:58:40.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.741+0000 7f641f400640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f640806ac00 0x7f640806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.741+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6414088710 con 0x7f64280ce930 2024-03-23T07:58:40.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.741+0000 7f641f400640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f6408070fa0 0x7f64080733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.741+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 --> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6408073ab0 con 0x7f6408070fa0 2024-03-23T07:58:40.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.741+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f641408acf0 con 0x7f64280ce930 2024-03-23T07:58:40.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.741+0000 7f6427400640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f640806ac00 0x7f640806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.742+0000 7f6427e00640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f6408070fa0 0x7f64080733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.742+0000 7f6427400640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f640806ac00 0x7f640806d0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6418036000 tx=0x7f6418008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.743+0000 7f6427e00640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f6408070fa0 0x7f64080733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.745+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== osd.7 v2:172.21.15.132:6816/37452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6408073ab0 con 0x7f6408070fa0 2024-03-23T07:58:40.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.759+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 --> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f64280cd1f0 con 0x7f6408070fa0 2024-03-23T07:58:40.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.760+0000 7f641f400640 1 -- 172.21.15.105:0/2810581487 <== osd.7 v2:172.21.15.132:6816/37452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6420006b50 con 0x7f6408070fa0 2024-03-23T07:58:40.761 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-23T07:58:40.761 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T07:58:40.761 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-23T07:58:40.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f6408070fa0 msgr2=0x7f64080733e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.132:6816/37452,v1:172.21.15.132:6818/37452] conn(0x7f6408070fa0 0x7f64080733e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f640806ac00 msgr2=0x7f640806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f640806ac00 0x7f640806d0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6418036000 tx=0x7f6418008030 comp rx=0 tx=0).stop 2024-03-23T07:58:40.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64280ce930 msgr2=0x7f64281542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 --2- 172.21.15.105:0/2810581487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64280ce930 0x7f64281542b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f641400c4d0 tx=0x7f641400c990 comp rx=0 tx=0).stop 2024-03-23T07:58:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f6427e00640 1 -- 172.21.15.105:0/2810581487 reap_dead start 2024-03-23T07:58:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 shutdown_connections 2024-03-23T07:58:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 >> 172.21.15.105:0/2810581487 conn(0x7f642806a0b0 msgr2=0x7f64281549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 shutdown_connections 2024-03-23T07:58:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.761+0000 7f642e200640 1 -- 172.21.15.105:0/2810581487 wait complete. 2024-03-23T07:58:40.778 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-23T07:58:40.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.981+0000 7f5c85200640 1 Processor -- start 2024-03-23T07:58:40.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.981+0000 7f5c85200640 1 -- start start 2024-03-23T07:58:40.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.982+0000 7f5c85200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 0x7f5c800b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.982+0000 7f5c85200640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8015e010 con 0x7f5c80159690 2024-03-23T07:58:40.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.982+0000 7f5c85200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800caec0 con 0x7f5c800b1e10 2024-03-23T07:58:40.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.982+0000 7f5c85200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800cc7f0 con 0x7f5c800cc330 2024-03-23T07:58:40.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.983+0000 7f5c7de00640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5c80159690 0x7f5c800ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59568/0 (socket says 172.21.15.105:59568) 2024-03-23T07:58:40.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.983+0000 7f5c7de00640 1 -- 172.21.15.105:0/2530417807 learned_addr learned my addr 172.21.15.105:0/2530417807 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:58:40.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.983+0000 7f5c7d400640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 0x7f5c800b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.983+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682061999 0 0) 0x7f5c8015e010 con 0x7f5c80159690 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.983+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c64005730 con 0x7f5c80159690 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813238139 0 0) 0x7f5c64005730 con 0x7f5c80159690 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 msgr2=0x7f5c800b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 0x7f5c800b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 >> v1:172.21.15.132:6789/0 conn(0x7f5c800b1e10 legacy=0x7f5c800cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800ccad0 con 0x7f5c80159690 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f5c70001670 con 0x7f5c80159690 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 >> v1:172.21.15.105:6789/0 conn(0x7f5c80159690 legacy=0x7f5c800ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 0x7f5c64005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c64006130 0x7f5c640069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006f10 0x7f5c64007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800cc7f0 con 0x7f5c64006130 2024-03-23T07:58:40.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6400dc70 con 0x7f5c64006f10 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6400de30 con 0x7f5c800cc330 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7d400640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c64006130 0x7f5c640069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.984+0000 7f5c7de00640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 0x7f5c64005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7e800640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006f10 0x7f5c64007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7d400640 1 -- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 msgr2=0x7f5c64005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7d400640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 0x7f5c64005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7d400640 1 -- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006f10 msgr2=0x7f5c64007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7d400640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006f10 0x7f5c64007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7d400640 1 -- 172.21.15.105:0/2530417807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64005730 con 0x7f5c64006130 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7e800640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006f10 0x7f5c64007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.985+0000 7f5c7de00640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800cc330 0x7f5c64005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.986+0000 7f5c7d400640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c64006130 0x7f5c640069d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009ac0 tx=0x7f5c64007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cc134c98de14b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.986+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c02c6b0 con 0x7f5c64006130 2024-03-23T07:58:40.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.986+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c02c810 con 0x7f5c64006130 2024-03-23T07:58:40.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.986+0000 7f5c7ca00640 1 -- 172.21.15.105:0/2530417807 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c02cba0 con 0x7f5c64006130 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.987+0000 7f5c85200640 1 -- 172.21.15.105:0/2530417807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c64006130 msgr2=0x7f5c640069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.987+0000 7f5c85200640 1 --2- 172.21.15.105:0/2530417807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c64006130 0x7f5c640069d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009ac0 tx=0x7f5c64007a00 comp rx=0 tx=0).stop 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.987+0000 7f5c7e800640 1 -- 172.21.15.105:0/2530417807 reap_dead start 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.987+0000 7f5c85200640 1 -- 172.21.15.105:0/2530417807 shutdown_connections 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.987+0000 7f5c85200640 1 -- 172.21.15.105:0/2530417807 >> 172.21.15.105:0/2530417807 conn(0x7f5c8006a0b0 msgr2=0x7f5c8006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.987+0000 7f5c85200640 1 -- 172.21.15.105:0/2530417807 shutdown_connections 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.987+0000 7f5c85200640 1 -- 172.21.15.105:0/2530417807 wait complete. 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.990+0000 7f5c85200640 1 Processor -- start 2024-03-23T07:58:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.990+0000 7f5c85200640 1 -- start start 2024-03-23T07:58:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006130 0x7f5c8014a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8014b570 0x7f5c801553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c8015b610 0x7f5c8015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80158b90 con 0x7f5c8014b570 2024-03-23T07:58:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8015bfd0 con 0x7f5c64006130 2024-03-23T07:58:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80157f80 con 0x7f5c8015b610 2024-03-23T07:58:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8014b570 0x7f5c801553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8014b570 0x7f5c801553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42896/0 (socket says 172.21.15.105:42896) 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 -- 172.21.15.105:0/599192045 learned_addr learned my addr 172.21.15.105:0/599192045 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7de00640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006130 0x7f5c8014a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7e800640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c8015b610 0x7f5c8015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 -- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c8015b610 msgr2=0x7f5c8015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c8015b610 0x7f5c8015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 -- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006130 msgr2=0x7f5c8014a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006130 0x7f5c8014a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 -- 172.21.15.105:0/599192045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c80158260 con 0x7f5c8014b570 2024-03-23T07:58:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c7d400640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8014b570 0x7f5c801553e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c02c250 tx=0x7f5c6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c76000640 1 -- 172.21.15.105:0/599192045 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c003810 con 0x7f5c8014b570 2024-03-23T07:58:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c76000640 1 -- 172.21.15.105:0/599192045 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c003970 con 0x7f5c8014b570 2024-03-23T07:58:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c80158540 con 0x7f5c8014b570 2024-03-23T07:58:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c76000640 1 -- 172.21.15.105:0/599192045 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c003ce0 con 0x7f5c8014b570 2024-03-23T07:58:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.991+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8014aca0 con 0x7f5c8014b570 2024-03-23T07:58:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.993+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c80073390 con 0x7f5c8014b570 2024-03-23T07:58:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.994+0000 7f5c76000640 1 -- 172.21.15.105:0/599192045 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5c6c047450 con 0x7f5c8014b570 2024-03-23T07:58:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.994+0000 7f5c76000640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5c6006ac00 0x7f5c6006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.994+0000 7f5c76000640 1 -- 172.21.15.105:0/599192045 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f5c6c0ab500 con 0x7f5c8014b570 2024-03-23T07:58:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.995+0000 7f5c7de00640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5c6006ac00 0x7f5c6006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:40.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.995+0000 7f5c7de00640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5c6006ac00 0x7f5c6006d0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c70001c90 tx=0x7f5c70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:40.999+0000 7f5c76000640 1 -- 172.21.15.105:0/599192045 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5c6c0768f0 con 0x7f5c8014b570 2024-03-23T07:58:41.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.194+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c800cd370 con 0x7f5c6006ac00 2024-03-23T07:58:41.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.195+0000 7f5c76000640 1 -- 172.21.15.105:0/599192045 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5c700109d0 con 0x7f5c6006ac00 2024-03-23T07:58:41.195 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 7 to deep-scrub 2024-03-23T07:58:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5c6006ac00 msgr2=0x7f5c6006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5c6006ac00 0x7f5c6006d0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c70001c90 tx=0x7f5c70009040 comp rx=0 tx=0).stop 2024-03-23T07:58:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8014b570 msgr2=0x7f5c801553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8014b570 0x7f5c801553e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c02c250 tx=0x7f5c6c004730 comp rx=0 tx=0).stop 2024-03-23T07:58:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 shutdown_connections 2024-03-23T07:58:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f5c6006ac00 0x7f5c6006d0c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c8015b610 0x7f5c8015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8014b570 0x7f5c801553e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 --2- 172.21.15.105:0/599192045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c64006130 0x7f5c8014a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 >> 172.21.15.105:0/599192045 conn(0x7f5c8006a0b0 msgr2=0x7f5c800b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:41.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 shutdown_connections 2024-03-23T07:58:41.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.198+0000 7f5c85200640 1 -- 172.21.15.105:0/599192045 wait complete. 2024-03-23T07:58:41.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:58:41.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.423+0000 7efe20200640 1 Processor -- start 2024-03-23T07:58:41.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.424+0000 7efe20200640 1 -- start start 2024-03-23T07:58:41.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.425+0000 7efe20200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe1815a940 0x7efe18099450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:41.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.425+0000 7efe20200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe180cc320 0x7efe180c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.425+0000 7efe20200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180cba30 con 0x7efe180cbec0 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.425+0000 7efe20200640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1815ad20 con 0x7efe180cc320 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.425+0000 7efe20200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180cb140 con 0x7efe1815a940 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.425+0000 7efe1e200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe180cc320 0x7efe180c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.425+0000 7efe1e200640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe180cc320 0x7efe180c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42910/0 (socket says 172.21.15.105:42910) 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.426+0000 7efe1e200640 1 -- 172.21.15.105:0/2363431073 learned_addr learned my addr 172.21.15.105:0/2363431073 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.426+0000 7efe1d800640 1 --2- 172.21.15.105:0/2363431073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe1815a940 0x7efe18099450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.426+0000 7efe1e200640 1 -- 172.21.15.105:0/2363431073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe1815a940 msgr2=0x7efe18099450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.426+0000 7efe1e200640 1 --2- 172.21.15.105:0/2363431073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe1815a940 0x7efe18099450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.426+0000 7efe1e200640 1 -- 172.21.15.105:0/2363431073 >> v1:172.21.15.132:6789/0 conn(0x7efe180cbec0 legacy=0x7efe180c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.426+0000 7efe1e200640 1 -- 172.21.15.105:0/2363431073 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18099ae0 con 0x7efe180cc320 2024-03-23T07:58:41.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.427+0000 7efe1e200640 1 --2- 172.21.15.105:0/2363431073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe180cc320 0x7efe180c4890 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efe10009870 tx=0x7efe1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db422d943e7bf7e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:41.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.427+0000 7efe17e00640 1 -- 172.21.15.105:0/2363431073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10030840 con 0x7efe180cc320 2024-03-23T07:58:41.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.427+0000 7efe17e00640 1 -- 172.21.15.105:0/2363431073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe100309a0 con 0x7efe180cc320 2024-03-23T07:58:41.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.427+0000 7efe17e00640 1 -- 172.21.15.105:0/2363431073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10030d30 con 0x7efe180cc320 2024-03-23T07:58:41.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 -- 172.21.15.105:0/2363431073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe180cc320 msgr2=0x7efe180c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 --2- 172.21.15.105:0/2363431073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe180cc320 0x7efe180c4890 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efe10009870 tx=0x7efe1002b980 comp rx=0 tx=0).stop 2024-03-23T07:58:41.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 -- 172.21.15.105:0/2363431073 shutdown_connections 2024-03-23T07:58:41.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 --2- 172.21.15.105:0/2363431073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe180cc320 0x7efe180c4890 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 --2- 172.21.15.105:0/2363431073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe1815a940 0x7efe18099450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 -- 172.21.15.105:0/2363431073 >> 172.21.15.105:0/2363431073 conn(0x7efe180b2120 msgr2=0x7efe180b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:41.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 -- 172.21.15.105:0/2363431073 shutdown_connections 2024-03-23T07:58:41.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.428+0000 7efe20200640 1 -- 172.21.15.105:0/2363431073 wait complete. 2024-03-23T07:58:41.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.430+0000 7efe20200640 1 Processor -- start 2024-03-23T07:58:41.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.430+0000 7efe20200640 1 -- start start 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe20200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe180cbec0 0x7efe180c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe20200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe1815a940 0x7efe18156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe20200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe180c7700 0x7efe180c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe20200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c8e60 con 0x7efe1815a940 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe20200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c81c0 con 0x7efe180c7700 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe20200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c84a0 con 0x7efe180cbec0 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe1ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe1815a940 0x7efe18156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe1ce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe1815a940 0x7efe18156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42916/0 (socket says 172.21.15.105:42916) 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe1ce00640 1 -- 172.21.15.105:0/1812186022 learned_addr learned my addr 172.21.15.105:0/1812186022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:58:41.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.431+0000 7efe1d800640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe180cbec0 0x7efe180c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe1e200640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe180c7700 0x7efe180c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe1ce00640 1 -- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe180cbec0 msgr2=0x7efe180c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe1ce00640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe180cbec0 0x7efe180c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe1ce00640 1 -- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe180c7700 msgr2=0x7efe180c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe1ce00640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe180c7700 0x7efe180c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe1ce00640 1 -- 172.21.15.105:0/1812186022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100094d0 con 0x7efe1815a940 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe1ce00640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe1815a940 0x7efe18156f80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c870 tx=0x7efe0800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe15600640 1 -- 172.21.15.105:0/1812186022 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08003b70 con 0x7efe1815a940 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe15600640 1 -- 172.21.15.105:0/1812186022 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08003cf0 con 0x7efe1815a940 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe15600640 1 -- 172.21.15.105:0/1812186022 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0800aa90 con 0x7efe1815a940 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe180afe50 con 0x7efe1815a940 2024-03-23T07:58:41.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.432+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe180afb20 con 0x7efe1815a940 2024-03-23T07:58:41.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.434+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18066bb0 con 0x7efe1815a940 2024-03-23T07:58:41.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.435+0000 7efe15600640 1 -- 172.21.15.105:0/1812186022 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efe0800abf0 con 0x7efe1815a940 2024-03-23T07:58:41.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.436+0000 7efe15600640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efdfc06ac00 0x7efdfc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:58:41.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.436+0000 7efe15600640 1 -- 172.21.15.105:0/1812186022 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7efe08089170 con 0x7efe1815a940 2024-03-23T07:58:41.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.436+0000 7efe1d800640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efdfc06ac00 0x7efdfc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:58:41.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.437+0000 7efe1d800640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efdfc06ac00 0x7efdfc06d0c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009a30 tx=0x7efe0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:58:41.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.441+0000 7efe15600640 1 -- 172.21.15.105:0/1812186022 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efe08054410 con 0x7efe1815a940 2024-03-23T07:58:41.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.632+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe180afce0 con 0x7efdfc06ac00 2024-03-23T07:58:41.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.634+0000 7efe15600640 1 -- 172.21.15.105:0/1812186022 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48371 (secure 0 0 0) 0x7efe180afce0 con 0x7efdfc06ac00 2024-03-23T07:58:41.635 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:58:41.636 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:58:41.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efdfc06ac00 msgr2=0x7efdfc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efdfc06ac00 0x7efdfc06d0c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009a30 tx=0x7efe0c008040 comp rx=0 tx=0).stop 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe1815a940 msgr2=0x7efe18156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe1815a940 0x7efe18156f80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c870 tx=0x7efe0800cd30 comp rx=0 tx=0).stop 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 shutdown_connections 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe180c7700 0x7efe180c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe1815a940 0x7efe18156f80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7efdfc06ac00 0x7efdfc06d0c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 --2- 172.21.15.105:0/1812186022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe180cbec0 0x7efe180c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 >> 172.21.15.105:0/1812186022 conn(0x7efe180b2120 msgr2=0x7efe18158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 shutdown_connections 2024-03-23T07:58:41.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:58:41.637+0000 7efe20200640 1 -- 172.21.15.105:0/1812186022 wait complete. 2024-03-23T07:58:41.655 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-03-23T07:58:40.088500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":221608,"kb_used_data":5416,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753112,"statfs":{"total":773094113280,"available":772867186688,"internally_reserved":0,"allocated":5545984,"data_stored":3070064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002215"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":187,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-03-23T07:58:30.402142+0000","last_change":"2024-03-23T07:56:07.403984+0000","last_active":"2024-03-23T07:58:30.402142+0000","last_peered":"2024-03-23T07:58:30.402142+0000","last_clean":"2024-03-23T07:58:30.402142+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:58:30.402142+0000","last_undegraded":"2024-03-23T07:58:30.402142+0000","last_fullsized":"2024-03-23T07:58:30.402142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T07:56:06.383438+0000","last_clean_scrub_stamp":"2024-03-23T07:56:06.383438+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476769,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"min":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3120000000000001,"5min":1.506,"15min":1.506},"last":0.72999999999999998}]},{"osd":1,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.872,"15min":0.88},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3140000000000001,"5min":1.827,"15min":1.827},"last":0.68100000000000005},{"interface":"front","average":{"1min":1,"5min":1.0049999999999999,"15min":1.006},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.46,"5min":1.861,"15min":1.861},"last":1.0389999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.498,"5min":1.944,"15min":1.944},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.093,"15min":1.1040000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.405,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.103}]},{"osd":4,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2490000000000001,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6910000000000001,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.383,"15min":1.4079999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.591,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.72}]},{"osd":5,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.466,"15min":1.498},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.6419999999999999,"5min":3.589,"15min":3.589},"last":1.361},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.389,"15min":1.415},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.734,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.585}]},{"osd":6,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.885,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.514,"15min":1.5469999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7130000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.474}]},{"osd":7,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.389,"15min":1.399},"min":{"1min":0.78100000000000003,"5min":0.753,"15min":0.753},"max":{"1min":1.8160000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.534},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7909999999999999,"5min":2.492,"15min":2.492},"last":1.399}]}]},{"osd":2,"up_from":16,"seq":68719476769,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89800000000000002,"15min":0.91400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5229999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.94399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.755}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.077,"15min":1.117},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1930000000000001,"5min":2.04,"15min":2.04},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.129,"15min":1.159},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.661,"5min":1.839,"15min":1.839},"last":1.552}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.056,"15min":1.085},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7689999999999999,"5min":2.081,"15min":2.081},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.288,"15min":1.331},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.585,"5min":2.012,"15min":2.012},"last":1.2470000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.4239999999999999,"15min":1.458},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8560000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.3959999999999999,"15min":1.4279999999999999},"min":{"1min":0.67200000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9279999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.4990000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4610000000000001},"min":{"1min":0.89500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.4990000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.571,"15min":1.609},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9470000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":1.367}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.504,"15min":1.5449999999999999},"min":{"1min":0.85299999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0249999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.5},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9610000000000001,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.46}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.522,"15min":1.556},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.1299999999999999,"5min":3.069,"15min":3.069},"last":1.2},{"interface":"front","average":{"1min":1.232,"5min":1.544,"15min":1.5960000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.0750000000000002,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.3939999999999999}]}]},{"osd":1,"up_from":16,"seq":68719476769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1138688,"data_stored":826534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3340000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.45,"5min":3.27,"15min":3.27},"last":0.85399999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.96799999999999997,"15min":0.98099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2110000000000001,"5min":2.758,"15min":2.758},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0840000000000001,"15min":1.1100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.55,"5min":3.1699999999999999,"15min":3.1699999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.163,"15min":1.1950000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.405,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.014,"5min":1.1459999999999999,"15min":1.1679999999999999},"min":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3029999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.0069999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.6080000000000001,"15min":1.6739999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6870000000000001,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.347,"15min":1.377},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5589999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.96099999999999997}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5569999999999999,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.254},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.411,"15min":1.4350000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.869,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.105}]},{"osd":6,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.5580000000000001,"15min":1.6080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6870000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.129},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4790000000000001,"15min":1.5129999999999999},"min":{"1min":0.84799999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7829999999999999,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.3280000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.6140000000000001,"15min":1.6639999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.75,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.472},{"interface":"front","average":{"1min":1.339,"5min":1.512,"15min":1.5409999999999999},"min":{"1min":0.85899999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.9650000000000001,"5min":3.145,"15min":3.145},"last":1.2410000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476771,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1359999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.514,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.85799999999999998}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4159999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.93700000000000006}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.97299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3320000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.5},{"interface":"front","average":{"1min":1.052,"5min":1.153,"15min":1.1699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4970000000000001,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.98099999999999998}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.238,"15min":1.244},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7070000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.8169999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.3089999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.294,"15min":1.304},"min":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9590000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.3440000000000001,"15min":1.359},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9059999999999999,"5min":2.867,"15min":2.867},"last":1.1659999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3939999999999999,"15min":1.4019999999999999},"min":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6759999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.3420000000000001,"15min":1.361},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6719999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.28}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.355,"15min":1.373},"min":{"1min":0.89000000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7669999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3919999999999999,"15min":1.4139999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7130000000000001,"5min":2.411,"15min":2.411},"last":1.5489999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.044,"15min":1.05},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.107,"15min":1.115},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.97199999999999998}]},{"osd":1,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2030000000000001,"15min":1.2230000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5189999999999999,"5min":1.893,"15min":1.893},"last":5.5549999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.1559999999999999,"15min":1.1879999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.2809999999999999,"15min":1.3120000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5600000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":5.5099999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.2989999999999999,"15min":1.3080000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.661,"5min":1.996,"15min":1.996},"last":1.0589999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.581,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.22,"5min":1.377,"15min":1.403},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6779999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.016}]},{"osd":4,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.351},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.784,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7150000000000001,"5min":2.198,"15min":2.198},"last":0.66500000000000004}]},{"osd":5,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.452,"15min":1.468},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.0350000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.51,"15min":1.526},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8420000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.591,"15min":1.6020000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.008,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.468,"5min":1.5660000000000001,"15min":1.5820000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.895,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.81499999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":0.99199999999999999,"15min":0.97099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.383},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0209999999999999,"15min":1.0229999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3660000000000001,"5min":1.387,"15min":1.387},"last":1.1839999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.746,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.776},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.159,"15min":1.1719999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8009999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.1040000000000001}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1819999999999999,"15min":1.1839999999999999},"min":{"1min":0.82899999999999996,"5min":0.624,"15min":0.624},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1599999999999999,"15min":1.1519999999999999},"min":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6200000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.153}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":0.97199999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":2.109},"last":1.474},{"interface":"front","average":{"1min":1.262,"5min":1.302,"15min":1.3089999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5589999999999999,"5min":1.968,"15min":1.968},"last":1.3480000000000001}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.3009999999999999},"min":{"1min":1.0209999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6639999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.411},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.296,"15min":1.284},"min":{"1min":1.0489999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9319999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.97599999999999998}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.446,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.073,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.466,"15min":1.468},"min":{"1min":1.1799999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":1.804,"5min":2.194,"15min":2.194},"last":1.528}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4910000000000001,"15min":1.5},"min":{"1min":1.1299999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.72,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.054},{"interface":"front","average":{"1min":1.508,"5min":1.546,"15min":1.552},"min":{"1min":1.2490000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.8899999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.3200000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0660000000000001,"15min":1.0669999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.246}]},{"osd":1,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.2989999999999999,"15min":1.339},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5269999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1759999999999999,"15min":1.194},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.038}]},{"osd":2,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1859999999999999,"15min":1.1970000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5629999999999999,"5min":1.915,"15min":1.915},"last":0.997},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.323,"15min":1.3560000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.478,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3}]},{"osd":3,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2370000000000001,"15min":1.254},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4379999999999999,"5min":1.734,"15min":1.734},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.387,"15min":1.4099999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.8240000000000001,"5min":1.988,"15min":1.988},"last":1.4650000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.347,"15min":1.365},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6379999999999999,"5min":2.198,"15min":2.198},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.383,"15min":1.413},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.081,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.5169999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.401},{"interface":"front","average":{"1min":1.389,"5min":1.581,"15min":1.613},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9650000000000001,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.575}]},{"osd":7,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5980000000000001,"15min":1.6200000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.121,"5min":2.403,"15min":2.403},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.52,"15min":1.548},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8720000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.4390000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1138688,"data_stored":826534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0660000000000001,"15min":1.0920000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.401,"15min":1.401},"last":1.321},{"interface":"front","average":{"1min":0.995,"5min":1.097,"15min":1.1140000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.98599999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1830000000000001,"15min":1.206},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5009999999999999,"5min":1.663,"15min":1.663},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1950000000000001,"15min":1.212},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5580000000000001,"5min":1.833,"15min":1.833},"last":1.0740000000000001}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3120000000000001,"15min":1.3220000000000001},"min":{"1min":0.78000000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.911,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1459999999999999,"15min":1.1619999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5449999999999999,"5min":1.72,"15min":1.72},"last":1.446}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.254,"15min":1.272},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.3009999999999999,"15min":1.3089999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8220000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.573}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.274,"15min":1.254},"min":{"1min":0.90500000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.377},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.351,"15min":1.369},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.589,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.456},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.3680000000000001,"15min":1.3819999999999999},"min":{"1min":0.72599999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.724,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.532}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4790000000000001,"15min":1.504},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8580000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.607},{"interface":"front","average":{"1min":1.512,"5min":1.5860000000000001,"15min":1.5980000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1200000000000001,"5min":2.282,"15min":2.282},"last":1.6359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:58:41.659 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-23T07:56:06.383438+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=23, tm_hour=7, tm_min=56, tm_sec=6, tm_wday=5, tm_yday=83, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=23, tm_hour=7, tm_min=58, tm_sec=34, tm_wday=5, tm_yday=83, tm_isdst=0) 2024-03-23T07:58:41.660 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-23T07:59:01.660 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T07:59:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.882+0000 7f3403e00640 1 Processor -- start 2024-03-23T07:59:01.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.882+0000 7f3403e00640 1 -- start start 2024-03-23T07:59:01.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.883+0000 7f3403e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 0x7f34040b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.883+0000 7f3403e00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34040cae50 0x7f34040cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.883+0000 7f3403e00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34041575b0 con 0x7f34040cdc00 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.883+0000 7f3403e00640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040cb820 con 0x7f34040cae50 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.883+0000 7f3403e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3404159f20 con 0x7f34041571d0 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3403400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34040cae50 0x7f34040cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3403400640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34040cae50 0x7f34040cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58648/0 (socket says 172.21.15.105:58648) 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3403400640 1 -- 172.21.15.105:0/3694158544 learned_addr learned my addr 172.21.15.105:0/3694158544 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3402000640 1 --1- 172.21.15.105:0/3694158544 >> v1:172.21.15.105:6789/0 conn(0x7f34040cdc00 0x7f340415aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42326/0 (socket says 172.21.15.105:42326) 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3403400640 1 -- 172.21.15.105:0/3694158544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 msgr2=0x7f34040b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3403400640 1 --2- 172.21.15.105:0/3694158544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 0x7f34040b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3403400640 1 -- 172.21.15.105:0/3694158544 >> v1:172.21.15.105:6789/0 conn(0x7f34040cdc00 legacy=0x7f340415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.884+0000 7f3403400640 1 -- 172.21.15.105:0/3694158544 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340415a200 con 0x7f34040cae50 2024-03-23T07:59:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.885+0000 7f3403400640 1 --2- 172.21.15.105:0/3694158544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f34040cb2e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009e50 tx=0x7f33fc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee1b8684ea02c68 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3401600640 1 -- 172.21.15.105:0/3694158544 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc02cb00 con 0x7f34040cae50 2024-03-23T07:59:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3401600640 1 -- 172.21.15.105:0/3694158544 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc02cc60 con 0x7f34040cae50 2024-03-23T07:59:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3401600640 1 -- 172.21.15.105:0/3694158544 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0343f0 con 0x7f34040cae50 2024-03-23T07:59:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 -- 172.21.15.105:0/3694158544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 msgr2=0x7f34040cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 --2- 172.21.15.105:0/3694158544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f34040cb2e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009e50 tx=0x7f33fc02bf60 comp rx=0 tx=0).stop 2024-03-23T07:59:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 -- 172.21.15.105:0/3694158544 shutdown_connections 2024-03-23T07:59:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 --2- 172.21.15.105:0/3694158544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f34040cb2e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 --2- 172.21.15.105:0/3694158544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 0x7f34040b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 -- 172.21.15.105:0/3694158544 >> 172.21.15.105:0/3694158544 conn(0x7f3404065df0 msgr2=0x7f3404069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:59:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 -- 172.21.15.105:0/3694158544 shutdown_connections 2024-03-23T07:59:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.886+0000 7f3403e00640 1 -- 172.21.15.105:0/3694158544 wait complete. 2024-03-23T07:59:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.888+0000 7f3403e00640 1 Processor -- start 2024-03-23T07:59:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.888+0000 7f3403e00640 1 -- start start 2024-03-23T07:59:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3403e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f3404141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3403e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34040cdc00 0x7f34040b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3403e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 0x7f34040ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3403e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404141ff0 con 0x7f34040cae50 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3403e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34041422d0 con 0x7f34040cdc00 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3403e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34041425b0 con 0x7f34041571d0 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3402a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f3404141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3402a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f3404141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58664/0 (socket says 172.21.15.105:58664) 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.889+0000 7f3402a00640 1 -- 172.21.15.105:0/687346630 learned_addr learned my addr 172.21.15.105:0/687346630 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3403400640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 0x7f34040ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3402000640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34040cdc00 0x7f34040b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3402a00640 1 -- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 msgr2=0x7f34040ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3402a00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 0x7f34040ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3402a00640 1 -- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34040cdc00 msgr2=0x7f34040b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3402a00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34040cdc00 0x7f34040b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3402a00640 1 -- 172.21.15.105:0/687346630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc009b30 con 0x7f34040cae50 2024-03-23T07:59:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f3402a00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f3404141a10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f33f400b4a0 tx=0x7f33f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.890+0000 7f33faa00640 1 -- 172.21.15.105:0/687346630 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4010610 con 0x7f34040cae50 2024-03-23T07:59:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.891+0000 7f33faa00640 1 -- 172.21.15.105:0/687346630 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4010770 con 0x7f34040cae50 2024-03-23T07:59:01.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.891+0000 7f33faa00640 1 -- 172.21.15.105:0/687346630 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4010b20 con 0x7f34040cae50 2024-03-23T07:59:01.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.891+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3404142890 con 0x7f34040cae50 2024-03-23T07:59:01.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.891+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34040b0e10 con 0x7f34040cae50 2024-03-23T07:59:01.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.892+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340404e030 con 0x7f34040cae50 2024-03-23T07:59:01.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.893+0000 7f33faa00640 1 -- 172.21.15.105:0/687346630 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f33f4010c80 con 0x7f34040cae50 2024-03-23T07:59:01.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.894+0000 7f33faa00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f33e406ac30 0x7f33e406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:01.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.894+0000 7f33faa00640 1 -- 172.21.15.105:0/687346630 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f33f4088ec0 con 0x7f34040cae50 2024-03-23T07:59:01.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.894+0000 7f3402000640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f33e406ac30 0x7f33e406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:01.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.895+0000 7f3402000640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f33e406ac30 0x7f33e406d0f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f33f0007fd0 tx=0x7f33f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:01.899+0000 7f33faa00640 1 -- 172.21.15.105:0/687346630 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f33f4054160 con 0x7f34040cae50 2024-03-23T07:59:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.083+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 --> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3404142720 con 0x7f33e406ac30 2024-03-23T07:59:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.089+0000 7f33faa00640 1 -- 172.21.15.105:0/687346630 <== mgr.4101 v2:172.21.15.105:6832/37091 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48324 (secure 0 0 0) 0x7f3404142720 con 0x7f33e406ac30 2024-03-23T07:59:02.089 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:59:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-23T07:59:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.091+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f33e406ac30 msgr2=0x7f33e406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.091+0000 7f3403e00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f33e406ac30 0x7f33e406d0f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f33f0007fd0 tx=0x7f33f0009040 comp rx=0 tx=0).stop 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.091+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 msgr2=0x7f3404141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.091+0000 7f3403e00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f3404141a10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f33f400b4a0 tx=0x7f33f400b960 comp rx=0 tx=0).stop 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 shutdown_connections 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f33e406ac30 0x7f33e406d0f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34041571d0 0x7f34040ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34040cdc00 0x7f34040b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 --2- 172.21.15.105:0/687346630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34040cae50 0x7f3404141a10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 >> 172.21.15.105:0/687346630 conn(0x7f3404065df0 msgr2=0x7f340415d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 shutdown_connections 2024-03-23T07:59:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.092+0000 7f3403e00640 1 -- 172.21.15.105:0/687346630 wait complete. 2024-03-23T07:59:02.109 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-03-23T07:59:00.093892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":221592,"kb_used_data":5416,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753128,"statfs":{"total":773094113280,"available":772867203072,"internally_reserved":0,"allocated":5545984,"data_stored":3070064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003379"},"pg_stats":[{"pgid":"1.0","version":"47'119","reported_seq":200,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-03-23T07:58:37.140357+0000","last_change":"2024-03-23T07:58:37.140513+0000","last_active":"2024-03-23T07:58:37.140357+0000","last_peered":"2024-03-23T07:58:37.140357+0000","last_clean":"2024-03-23T07:58:37.140357+0000","last_became_active":"2024-03-23T07:56:07.403276+0000","last_became_peered":"2024-03-23T07:56:07.403276+0000","last_unstale":"2024-03-23T07:58:37.140357+0000","last_undegraded":"2024-03-23T07:58:37.140357+0000","last_fullsized":"2024-03-23T07:58:37.140357+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'119","last_scrub_stamp":"2024-03-23T07:58:37.140307+0000","last_deep_scrub":"47'119","last_deep_scrub_stamp":"2024-03-23T07:58:37.140307+0000","last_clean_scrub_stamp":"2024-03-23T07:58:37.140307+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:54:01.462288+0000","scrub_duration":0.010116099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"min":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3120000000000001,"5min":1.506,"15min":1.506},"last":0.81299999999999994}]},{"osd":1,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.872,"15min":0.88},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3140000000000001,"5min":1.827,"15min":1.827},"last":0.96199999999999997},{"interface":"front","average":{"1min":1,"5min":1.0049999999999999,"15min":1.006},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.46,"5min":1.861,"15min":1.861},"last":0.878}]},{"osd":2,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.498,"5min":1.944,"15min":1.944},"last":1.52},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.093,"15min":1.1040000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.405,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.177}]},{"osd":4,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2490000000000001,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6910000000000001,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.383,"15min":1.4079999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.591,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.204}]},{"osd":5,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.466,"15min":1.498},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.6419999999999999,"5min":3.589,"15min":3.589},"last":1.038},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.389,"15min":1.415},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.734,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.0009999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.885,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.514,"15min":1.5469999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7130000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.266}]},{"osd":7,"last update":"Sat Mar 23 07:58:13 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.389,"15min":1.399},"min":{"1min":0.78100000000000003,"5min":0.753,"15min":0.753},"max":{"1min":1.8160000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7909999999999999,"5min":2.492,"15min":2.492},"last":1.429}]}]},{"osd":2,"up_from":16,"seq":68719476773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89800000000000002,"15min":0.91400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5229999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.94399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.0449999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.077,"15min":1.117},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1930000000000001,"5min":2.04,"15min":2.04},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.129,"15min":1.159},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.661,"5min":1.839,"15min":1.839},"last":0.92700000000000005}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.056,"15min":1.085},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7689999999999999,"5min":2.081,"15min":2.081},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.288,"15min":1.331},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.585,"5min":2.012,"15min":2.012},"last":1.542}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.4239999999999999,"15min":1.458},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8560000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.357},{"interface":"front","average":{"1min":1.206,"5min":1.3959999999999999,"15min":1.4279999999999999},"min":{"1min":0.67200000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9279999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.1499999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.4159999999999999,"15min":1.4610000000000001},"min":{"1min":0.89500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.4990000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.571,"15min":1.609},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9470000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":1.26}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.504,"15min":1.5449999999999999},"min":{"1min":0.85299999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0249999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.429},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.5},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9610000000000001,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.1879999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.522,"15min":1.556},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.1299999999999999,"5min":3.069,"15min":3.069},"last":1.593},{"interface":"front","average":{"1min":1.232,"5min":1.544,"15min":1.5960000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.0750000000000002,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.462}]}]},{"osd":1,"up_from":16,"seq":68719476773,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1138688,"data_stored":826534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3340000000000001,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.45,"5min":3.27,"15min":3.27},"last":0.79700000000000004}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.96799999999999997,"15min":0.98099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2110000000000001,"5min":2.758,"15min":2.758},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0840000000000001,"15min":1.1100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.55,"5min":3.1699999999999999,"15min":3.1699999999999999},"last":1.077}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.163,"15min":1.1950000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.405,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.1459999999999999,"15min":1.1679999999999999},"min":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3029999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.409}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.6080000000000001,"15min":1.6739999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6870000000000001,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":1.038},{"interface":"front","average":{"1min":1.165,"5min":1.347,"15min":1.377},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5589999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.3779999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5569999999999999,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":1.306},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.411,"15min":1.4350000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.869,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.4550000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.5580000000000001,"15min":1.6080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6870000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.226},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4790000000000001,"15min":1.5129999999999999},"min":{"1min":0.84799999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7829999999999999,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.4419999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.6140000000000001,"15min":1.6639999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.75,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.512,"15min":1.5409999999999999},"min":{"1min":0.85899999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.9650000000000001,"5min":3.145,"15min":3.145},"last":1.552}]}]},{"osd":0,"up_from":16,"seq":68719476775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1359999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.514,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.1890000000000001}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4159999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86499999999999999,"15min":0.86299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.81499999999999995}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.97299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3320000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.88},{"interface":"front","average":{"1min":1.052,"5min":1.153,"15min":1.1699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4970000000000001,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.0609999999999999}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.238,"15min":1.244},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7070000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.8169999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.2629999999999999}]},{"osd":5,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.294,"15min":1.304},"min":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9590000000000001,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":1.151},{"interface":"front","average":{"1min":1.252,"5min":1.3440000000000001,"15min":1.359},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9059999999999999,"5min":2.867,"15min":2.867},"last":1.5529999999999999}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3939999999999999,"15min":1.4019999999999999},"min":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6759999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.3420000000000001,"15min":1.361},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6719999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4359999999999999}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.355,"15min":1.373},"min":{"1min":0.89000000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7669999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.363},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3919999999999999,"15min":1.4139999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7130000000000001,"5min":2.411,"15min":2.411},"last":1.458}]}]},{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.044,"15min":1.05},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.107,"15min":1.115},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.1699999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2030000000000001,"15min":1.2230000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5189999999999999,"5min":1.893,"15min":1.893},"last":1.427},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.1559999999999999,"15min":1.1879999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.90700000000000003}]},{"osd":2,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.2809999999999999,"15min":1.3120000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5600000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.2989999999999999,"15min":1.3080000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.661,"5min":1.996,"15min":1.996},"last":1.0980000000000001}]},{"osd":3,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.581,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.242},{"interface":"front","average":{"1min":1.22,"5min":1.377,"15min":1.403},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6779999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.474}]},{"osd":4,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.351},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.784,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.3620000000000001,"15min":1.3899999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7150000000000001,"5min":2.198,"15min":2.198},"last":1.024}]},{"osd":5,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.452,"15min":1.468},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.0350000000000001,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.363},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.51,"15min":1.526},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8420000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.5580000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:58:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.591,"15min":1.6020000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.008,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.288},{"interface":"front","average":{"1min":1.468,"5min":1.5660000000000001,"15min":1.5820000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.895,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.2689999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":0.99199999999999999,"15min":0.97099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0209999999999999,"15min":1.0229999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3660000000000001,"5min":1.387,"15min":1.387},"last":1.2310000000000001}]},{"osd":1,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.746,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.776},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.159,"15min":1.1719999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8009999999999999,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1819999999999999,"15min":1.1839999999999999},"min":{"1min":0.82899999999999996,"5min":0.624,"15min":0.624},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1599999999999999,"15min":1.1519999999999999},"min":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6200000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.6879999999999999}]},{"osd":3,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":0.97199999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":2.109},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.302,"15min":1.3089999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.5589999999999999,"5min":1.968,"15min":1.968},"last":1.504}]},{"osd":4,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.3009999999999999},"min":{"1min":1.0209999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.6639999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.296,"15min":1.284},"min":{"1min":1.0489999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9319999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.609}]},{"osd":5,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.446,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.073,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.571},{"interface":"front","average":{"1min":1.45,"5min":1.466,"15min":1.468},"min":{"1min":1.1799999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":1.804,"5min":2.194,"15min":2.194},"last":0.88900000000000001}]},{"osd":7,"last update":"Sat Mar 23 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4910000000000001,"15min":1.5},"min":{"1min":1.1299999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.72,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.468},{"interface":"front","average":{"1min":1.508,"5min":1.546,"15min":1.552},"min":{"1min":1.2490000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.8899999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.4359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":236166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0660000000000001,"15min":1.0669999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.89500000000000002}]},{"osd":1,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.2989999999999999,"15min":1.339},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5269999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1759999999999999,"15min":1.194},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.0169999999999999}]},{"osd":2,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1859999999999999,"15min":1.1970000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5629999999999999,"5min":1.915,"15min":1.915},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.323,"15min":1.3560000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.478,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.208}]},{"osd":3,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2370000000000001,"15min":1.254},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4379999999999999,"5min":1.734,"15min":1.734},"last":1.155},{"interface":"front","average":{"1min":1.248,"5min":1.387,"15min":1.4099999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.8240000000000001,"5min":1.988,"15min":1.988},"last":1.3360000000000001}]},{"osd":5,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.347,"15min":1.365},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6379999999999999,"5min":2.198,"15min":2.198},"last":1.474},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.383,"15min":1.413},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.081,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.1200000000000001}]},{"osd":6,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.581,"15min":1.613},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9650000000000001,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.53}]},{"osd":7,"last update":"Sat Mar 23 07:58:10 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5980000000000001,"15min":1.6200000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.121,"5min":2.403,"15min":2.403},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.52,"15min":1.548},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8720000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.2889999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1138688,"data_stored":826534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0660000000000001,"15min":1.0920000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.401,"15min":1.401},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.995,"5min":1.097,"15min":1.1140000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1339999999999999}]},{"osd":1,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1830000000000001,"15min":1.206},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5009999999999999,"5min":1.663,"15min":1.663},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1950000000000001,"15min":1.212},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5580000000000001,"5min":1.833,"15min":1.833},"last":1.107}]},{"osd":2,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3120000000000001,"15min":1.3220000000000001},"min":{"1min":0.78000000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.911,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1459999999999999,"15min":1.1619999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5449999999999999,"5min":1.72,"15min":1.72},"last":1.1830000000000001}]},{"osd":3,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.254,"15min":1.272},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.3009999999999999,"15min":1.3089999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8220000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.24}]},{"osd":4,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.274,"15min":1.254},"min":{"1min":0.90500000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.319},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.351,"15min":1.369},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.589,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.347}]},{"osd":6,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.456},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.3680000000000001,"15min":1.3819999999999999},"min":{"1min":0.72599999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.724,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.258}]},{"osd":7,"last update":"Sat Mar 23 07:58:08 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4790000000000001,"15min":1.504},"min":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8580000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.512,"5min":1.5860000000000001,"15min":1.5980000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1200000000000001,"5min":2.282,"15min":2.282},"last":1.855}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T07:59:02.111 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-23T07:59:02.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.285+0000 7f9ef1c00640 1 Processor -- start 2024-03-23T07:59:02.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.286+0000 7f9ef1c00640 1 -- start start 2024-03-23T07:59:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.287+0000 7f9ef1c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9eec15c8f0 0x7f9eec15d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.287+0000 7f9ef1c00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec0cd210 con 0x7f9eec0b17d0 2024-03-23T07:59:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.287+0000 7f9ef1c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec15dea0 con 0x7f9eec0c8e50 2024-03-23T07:59:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.287+0000 7f9ef1c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eec0b2020 con 0x7f9eec15c8f0 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9ee3400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9eec0b17d0 0x7f9eec0c8740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42342/0 (socket says 172.21.15.105:42342) 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9ee3400640 1 -- 172.21.15.105:0/4248164172 learned_addr learned my addr 172.21.15.105:0/4248164172 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9eebe00640 1 --2- 172.21.15.105:0/4248164172 >> v2:172.21.15.132:3300/0 conn(0x7f9eec15c8f0 0x7f9eec15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453279648 0 0) 0x7f9eec0cd210 con 0x7f9eec0b17d0 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed0003520 con 0x7f9eec0b17d0 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3286596871 0 0) 0x7f9ed0003520 con 0x7f9eec0b17d0 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 >> v2:172.21.15.132:3300/0 conn(0x7f9eec15c8f0 msgr2=0x7f9eec15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9eeb400640 1 --2- 172.21.15.105:0/4248164172 >> v2:172.21.15.132:3300/0 conn(0x7f9eec15c8f0 0x7f9eec15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.288+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 >> v1:172.21.15.132:6789/0 conn(0x7f9eec0c8e50 legacy=0x7f9eec0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0b1d40 con 0x7f9eec0b17d0 2024-03-23T07:59:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7f9ed4001670 con 0x7f9eec0b17d0 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 >> v1:172.21.15.105:6789/0 conn(0x7f9eec0b17d0 legacy=0x7f9eec0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed0003970 0x7f9ed0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed00047c0 0x7f9ed0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed0005180 0x7f9ed000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0b2020 con 0x7f9ed0003970 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed000c020 con 0x7f9ed00047c0 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed000c1e0 con 0x7f9ed0005180 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eebe00640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed0003970 0x7f9ed0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eebe00640 1 -- 172.21.15.105:0/4248164172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed0005180 msgr2=0x7f9ed000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eebe00640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed0005180 0x7f9ed000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eebe00640 1 -- 172.21.15.105:0/4248164172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed00047c0 msgr2=0x7f9ed0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eebe00640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed00047c0 0x7f9ed0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9eebe00640 1 -- 172.21.15.105:0/4248164172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0003520 con 0x7f9ed0003970 2024-03-23T07:59:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.289+0000 7f9ef1200640 1 -- 172.21.15.105:0/4248164172 reap_dead start 2024-03-23T07:59:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.290+0000 7f9eebe00640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed0003970 0x7f9ed0003d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009fc0 tx=0x7f9ed00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba8d9de664f479f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc02cc60 con 0x7f9ed0003970 2024-03-23T07:59:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00f040 con 0x7f9ed0003970 2024-03-23T07:59:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9eeb400640 1 -- 172.21.15.105:0/4248164172 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc034440 con 0x7f9ed0003970 2024-03-23T07:59:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9ef1c00640 1 -- 172.21.15.105:0/4248164172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed0003970 msgr2=0x7f9ed0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed0003970 0x7f9ed0003d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009fc0 tx=0x7f9ed00057f0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9ef1c00640 1 -- 172.21.15.105:0/4248164172 shutdown_connections 2024-03-23T07:59:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/4248164172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed0003970 0x7f9ed0003d90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9ef1c00640 1 -- 172.21.15.105:0/4248164172 >> 172.21.15.105:0/4248164172 conn(0x7f9eec098a40 msgr2=0x7f9eec0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:59:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9ef1c00640 1 -- 172.21.15.105:0/4248164172 shutdown_connections 2024-03-23T07:59:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.291+0000 7f9ef1c00640 1 -- 172.21.15.105:0/4248164172 wait complete. 2024-03-23T07:59:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.293+0000 7f9ef1c00640 1 Processor -- start 2024-03-23T07:59:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.293+0000 7f9ef1c00640 1 -- start start 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed0003970 0x7f9eec1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eec158880 0x7f9eec1f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eec158d00 0x7f9eec1f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec15bab0 con 0x7f9eec158d00 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec144af0 con 0x7f9ed0003970 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec15b210 con 0x7f9eec158880 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eec158d00 0x7f9eec1f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eec158d00 0x7f9eec1f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58690/0 (socket says 172.21.15.105:58690) 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9ef1200640 1 -- 172.21.15.105:0/1309531252 learned_addr learned my addr 172.21.15.105:0/1309531252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.294+0000 7f9eebe00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed0003970 0x7f9eec1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ee3400640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eec158880 0x7f9eec1f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1200640 1 -- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eec158880 msgr2=0x7f9eec1f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1200640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eec158880 0x7f9eec1f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1200640 1 -- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed0003970 msgr2=0x7f9eec1445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1200640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed0003970 0x7f9eec1445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1200640 1 -- 172.21.15.105:0/1309531252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec15b4f0 con 0x7f9eec158d00 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1200640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eec158d00 0x7f9eec1f5d20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0b02e0 tx=0x7f9ee400c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4004220 con 0x7f9eec158d00 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee40043a0 con 0x7f9eec158d00 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec15b7d0 con 0x7f9eec158d00 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4007310 con 0x7f9eec158d00 2024-03-23T07:59:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.295+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec1f6540 con 0x7f9eec158d00 2024-03-23T07:59:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.297+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec0ad6f0 con 0x7f9eec158d00 2024-03-23T07:59:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.297+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ee4007470 con 0x7f9eec158d00 2024-03-23T07:59:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.298+0000 7f9ee8c00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ec806ac30 0x7f9ec806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.299+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f9ee4088170 con 0x7f9eec158d00 2024-03-23T07:59:02.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.299+0000 7f9eebe00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ec806ac30 0x7f9ec806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.299+0000 7f9eebe00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ec806ac30 0x7f9ec806d0f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9edc036000 tx=0x7f9edc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.303+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9eec0ad6f0 con 0x7f9eec158d00 2024-03-23T07:59:02.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.491+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f9eec0ad6f0 con 0x7f9eec158d00 2024-03-23T07:59:02.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.498+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9ee4053540 con 0x7f9eec158d00 2024-03-23T07:59:02.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.498+0000 7f9ee8c00640 1 -- 172.21.15.105:0/1309531252 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9eec0ad6f0 con 0x7f9eec158d00 2024-03-23T07:59:02.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.500+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ec806ac30 msgr2=0x7f9ec806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.500+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ec806ac30 0x7f9ec806d0f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9edc036000 tx=0x7f9edc02e000 comp rx=0 tx=0).stop 2024-03-23T07:59:02.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.500+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eec158d00 msgr2=0x7f9eec1f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.500+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eec158d00 0x7f9eec1f5d20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0b02e0 tx=0x7f9ee400c5f0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 shutdown_connections 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9eec158d00 0x7f9eec1f5d20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7f9ec806ac30 0x7f9ec806d0f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eec158880 0x7f9eec1f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 --2- 172.21.15.105:0/1309531252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed0003970 0x7f9eec1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 >> 172.21.15.105:0/1309531252 conn(0x7f9eec098a40 msgr2=0x7f9eec0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 shutdown_connections 2024-03-23T07:59:02.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.501+0000 7f9ef1c00640 1 -- 172.21.15.105:0/1309531252 wait complete. 2024-03-23T07:59:02.517 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T07:59:02.517 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T07:59:02.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.701+0000 7fc489a00640 1 Processor -- start 2024-03-23T07:59:02.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.701+0000 7fc489a00640 1 -- start start 2024-03-23T07:59:02.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.702+0000 7fc489a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc48415cda0 0x7fc4840cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.702+0000 7fc489a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 0x7fc4840b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.703+0000 7fc489a00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4840d1830 con 0x7fc4840b4350 2024-03-23T07:59:02.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.703+0000 7fc489a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4840b1f20 con 0x7fc48415cda0 2024-03-23T07:59:02.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.703+0000 7fc489a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840d0c20 con 0x7fc4840b5a30 2024-03-23T07:59:02.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.703+0000 7fc483400640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc4840b4350 0x7fc4840d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42358/0 (socket says 172.21.15.105:42358) 2024-03-23T07:59:02.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.703+0000 7fc483400640 1 -- 172.21.15.105:0/1644665572 learned_addr learned my addr 172.21.15.105:0/1644665572 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.703+0000 7fc482000640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 0x7fc4840b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.703+0000 7fc482a00640 1 --2- 172.21.15.105:0/1644665572 >> v2:172.21.15.132:3300/0 conn(0x7fc48415cda0 0x7fc4840cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635905193 0 0) 0x7fc4840d1830 con 0x7fc4840b4350 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc468003520 con 0x7fc4840b4350 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400102444 0 0) 0x7fc468003520 con 0x7fc4840b4350 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 msgr2=0x7fc4840b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 0x7fc4840b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 >> v2:172.21.15.132:3300/0 conn(0x7fc48415cda0 msgr2=0x7fc4840cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 --2- 172.21.15.105:0/1644665572 >> v2:172.21.15.132:3300/0 conn(0x7fc48415cda0 0x7fc4840cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4840d0f00 con 0x7fc4840b4350 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4274432189 0 0) 0x7fc47c0019f0 con 0x7fc4840b4350 2024-03-23T07:59:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 >> v1:172.21.15.105:6789/0 conn(0x7fc4840b4350 legacy=0x7fc4840d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 0x7fc468003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc468003ef0 0x7fc4680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc468004d00 0x7fc468005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840b1f20 con 0x7fc468004d00 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840d0c20 con 0x7fc468003ef0 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.704+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46800b980 con 0x7fc4840b5a30 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc483400640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc468004d00 0x7fc468005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc482a00640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 0x7fc468003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc482000640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc468003ef0 0x7fc4680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc483400640 1 -- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 msgr2=0x7fc468003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc483400640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4840b5a30 0x7fc468003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc483400640 1 -- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc468003ef0 msgr2=0x7fc4680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T07:59:02.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc483400640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc468003ef0 0x7fc4680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.705+0000 7fc483400640 1 -- 172.21.15.105:0/1644665572 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468003520 con 0x7fc468004d00 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.707+0000 7fc483400640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc468004d00 0x7fc468005180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00a860 tx=0x7fc4680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af27dad30681fad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.707+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c034630 con 0x7fc468004d00 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.707+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc47c02f070 con 0x7fc468004d00 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.707+0000 7fc481600640 1 -- 172.21.15.105:0/1644665572 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c034a50 con 0x7fc468004d00 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.708+0000 7fc489a00640 1 -- 172.21.15.105:0/1644665572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc468004d00 msgr2=0x7fc468005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.708+0000 7fc489a00640 1 --2- 172.21.15.105:0/1644665572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc468004d00 0x7fc468005180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00a860 tx=0x7fc4680057f0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.708+0000 7fc483400640 1 -- 172.21.15.105:0/1644665572 reap_dead start 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.708+0000 7fc489a00640 1 -- 172.21.15.105:0/1644665572 shutdown_connections 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.708+0000 7fc489a00640 1 -- 172.21.15.105:0/1644665572 >> 172.21.15.105:0/1644665572 conn(0x7fc4840cf350 msgr2=0x7fc4840cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.708+0000 7fc489a00640 1 -- 172.21.15.105:0/1644665572 shutdown_connections 2024-03-23T07:59:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.708+0000 7fc489a00640 1 -- 172.21.15.105:0/1644665572 wait complete. 2024-03-23T07:59:02.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.710+0000 7fc489a00640 1 Processor -- start 2024-03-23T07:59:02.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.710+0000 7fc489a00640 1 -- start start 2024-03-23T07:59:02.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.711+0000 7fc489a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4840c8040 0x7fc4840c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.711+0000 7fc489a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48414e510 0x7fc4841f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.711+0000 7fc489a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4841f19e0 0x7fc48414a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.711+0000 7fc489a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840c8960 con 0x7fc48414e510 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc489a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840c91b0 con 0x7fc4840c8040 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc489a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc484148610 con 0x7fc4841f19e0 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48414e510 0x7fc4841f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48414e510 0x7fc4841f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58712/0 (socket says 172.21.15.105:58712) 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 -- 172.21.15.105:0/1885674044 learned_addr learned my addr 172.21.15.105:0/1885674044 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 -- 172.21.15.105:0/1885674044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4841f19e0 msgr2=0x7fc48414a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4841f19e0 0x7fc48414a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 -- 172.21.15.105:0/1885674044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4840c8040 msgr2=0x7fc4840c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4840c8040 0x7fc4840c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 -- 172.21.15.105:0/1885674044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4840c6770 con 0x7fc48414e510 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.712+0000 7fc482000640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48414e510 0x7fc4841f14a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc470011570 tx=0x7fc470011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.713+0000 7fc47aa00640 1 -- 172.21.15.105:0/1885674044 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470015930 con 0x7fc48414e510 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.713+0000 7fc47aa00640 1 -- 172.21.15.105:0/1885674044 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc470015a90 con 0x7fc48414e510 2024-03-23T07:59:02.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.713+0000 7fc47aa00640 1 -- 172.21.15.105:0/1885674044 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470015e80 con 0x7fc48414e510 2024-03-23T07:59:02.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.713+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4840c8b60 con 0x7fc48414e510 2024-03-23T07:59:02.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.713+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4840c6a50 con 0x7fc48414e510 2024-03-23T07:59:02.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.714+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4840d3700 con 0x7fc48414e510 2024-03-23T07:59:02.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.715+0000 7fc47aa00640 1 -- 172.21.15.105:0/1885674044 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc47001d400 con 0x7fc48414e510 2024-03-23T07:59:02.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.716+0000 7fc47aa00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46406ac30 0x7fc46406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T07:59:02.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.716+0000 7fc47aa00640 1 -- 172.21.15.105:0/1885674044 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc470089b20 con 0x7fc48414e510 2024-03-23T07:59:02.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.716+0000 7fc482a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46406ac30 0x7fc46406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T07:59:02.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.717+0000 7fc482a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46406ac30 0x7fc46406d0f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc48415ff20 tx=0x7fc474008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T07:59:02.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.721+0000 7fc47aa00640 1 -- 172.21.15.105:0/1885674044 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc470054d30 con 0x7fc48414e510 2024-03-23T07:59:02.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.942+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc484073390 con 0x7fc48414e510 2024-03-23T07:59:02.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.943+0000 7fc47aa00640 1 -- 172.21.15.105:0/1885674044 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc470022090 con 0x7fc48414e510 2024-03-23T07:59:02.943 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-23T07:59:02.943 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/632041556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":632041556},{"type":"v1","addr":"172.21.15.132:6835","nonce":632041556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2667628644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2667628644},{"type":"v1","addr":"172.21.15.132:6834","nonce":2667628644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/12048571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":12048571},{"type":"v1","addr":"172.21.15.105:6837","nonce":12048571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/912024571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":912024571},{"type":"v1","addr":"172.21.15.105:6835","nonce":912024571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[]} 2024-03-23T07:59:02.943 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 24 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46406ac30 msgr2=0x7fc46406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46406ac30 0x7fc46406d0f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc48415ff20 tx=0x7fc474008040 comp rx=0 tx=0).stop 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48414e510 msgr2=0x7fc4841f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48414e510 0x7fc4841f14a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc470011570 tx=0x7fc470011a30 comp rx=0 tx=0).stop 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 shutdown_connections 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:6832/37091,v1:172.21.15.105:6833/37091] conn(0x7fc46406ac30 0x7fc46406d0f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4841f19e0 0x7fc48414a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc48414e510 0x7fc4841f14a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 --2- 172.21.15.105:0/1885674044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4840c8040 0x7fc4840c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T07:59:02.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 >> 172.21.15.105:0/1885674044 conn(0x7fc4840cf350 msgr2=0x7fc4840b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T07:59:02.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 shutdown_connections 2024-03-23T07:59:02.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-23T07:59:02.946+0000 7fc489a00640 1 -- 172.21.15.105:0/1885674044 wait complete. 2024-03-23T07:59:02.969 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-23T07:59:02.969 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-23T07:59:02.969 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T07:59:02.969 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:02.987 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T07:59:02.987 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-23T07:59:02.987 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.050 INFO:tasks.ceph.mds.c:Stopped 2024-03-23T07:59:03.050 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T07:59:03.050 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.073 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T07:59:03.074 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-23T07:59:03.074 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.140 INFO:tasks.ceph.mds.d:Stopped 2024-03-23T07:59:03.141 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-23T07:59:03.141 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-23T07:59:03.141 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.202 INFO:tasks.ceph.osd.0:Stopped 2024-03-23T07:59:03.202 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-23T07:59:03.202 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.278 INFO:tasks.ceph.osd.1:Stopped 2024-03-23T07:59:03.278 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-23T07:59:03.278 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.336 INFO:tasks.ceph.osd.2:Stopped 2024-03-23T07:59:03.336 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-23T07:59:03.337 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.418 INFO:tasks.ceph.osd.3:Stopped 2024-03-23T07:59:03.419 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-23T07:59:03.419 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.494 INFO:tasks.ceph.osd.4:Stopped 2024-03-23T07:59:03.494 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-23T07:59:03.495 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.568 INFO:tasks.ceph.osd.5:Stopped 2024-03-23T07:59:03.568 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-23T07:59:03.568 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.622 INFO:tasks.ceph.osd.6:Stopped 2024-03-23T07:59:03.622 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-23T07:59:03.622 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.693 INFO:tasks.ceph.osd.7:Stopped 2024-03-23T07:59:03.694 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-23T07:59:03.694 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-23T07:59:03.694 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.767 INFO:tasks.ceph.mgr.y:Stopped 2024-03-23T07:59:03.767 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-23T07:59:03.767 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.843 INFO:tasks.ceph.mgr.x:Stopped 2024-03-23T07:59:03.844 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-23T07:59:03.844 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-23T07:59:03.844 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.886 INFO:tasks.ceph.mon.a:Stopped 2024-03-23T07:59:03.886 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-23T07:59:03.886 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.925 INFO:tasks.ceph.mon.b:Stopped 2024-03-23T07:59:03.926 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-23T07:59:03.926 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T07:59:03.996 INFO:tasks.ceph.mon.c:Stopped 2024-03-23T07:59:03.996 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-23T07:59:03.996 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds.a' | head -n 1 2024-03-23T07:59:04.098 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:59:04.098 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-23T07:59:04.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:59:04.364 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-23T07:59:04.505 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:59:04.505 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-23T07:59:04.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:59:04.625 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-23T07:59:04.773 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:59:04.773 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-23T07:59:05.033 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:59:05.033 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-23T07:59:05.174 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:59:05.174 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-23T07:59:05.327 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:59:05.328 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-23T07:59:05.477 INFO:tasks.ceph:Archiving mon data... 2024-03-23T07:59:05.478 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/data/mon.a.tgz 2024-03-23T07:59:05.479 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-03-23T07:59:05.534 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.L8VuIX79oS 2024-03-23T07:59:05.535 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.L8VuIX79oS 2024-03-23T07:59:05.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.L8VuIX79oS 2024-03-23T07:59:05.854 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.L8VuIX79oS is 316KB 2024-03-23T07:59:05.964 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.L8VuIX79oS 2024-03-23T07:59:06.018 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/data/mon.b.tgz 2024-03-23T07:59:06.019 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-03-23T07:59:06.073 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.9hciFZh55X 2024-03-23T07:59:06.074 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.9hciFZh55X 2024-03-23T07:59:06.284 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.9hciFZh55X 2024-03-23T07:59:06.431 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.9hciFZh55X is 417KB 2024-03-23T07:59:06.540 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.9hciFZh55X 2024-03-23T07:59:06.597 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/data/mon.c.tgz 2024-03-23T07:59:06.597 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-03-23T07:59:06.649 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.tn3VCvwmzF 2024-03-23T07:59:06.649 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tn3VCvwmzF 2024-03-23T07:59:06.836 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.tn3VCvwmzF 2024-03-23T07:59:06.980 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.tn3VCvwmzF is 417KB 2024-03-23T07:59:07.087 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.tn3VCvwmzF 2024-03-23T07:59:07.141 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-23T07:59:07.142 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-23T07:59:07.149 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-23T07:59:07.245 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-23T07:59:07.245 DEBUG:teuthology.orchestra.run.smithi105:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-23T07:59:07.252 DEBUG:teuthology.orchestra.run.smithi132:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-23T07:59:07.345 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-23T07:59:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-23T07:59:07.360 INFO:tasks.ceph:Archiving crash dumps... 2024-03-23T07:59:07.361 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/remote/smithi105/crash 2024-03-23T07:59:07.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-23T07:59:07.438 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/remote/smithi132/crash 2024-03-23T07:59:07.439 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-23T07:59:07.538 INFO:tasks.ceph:Shutting down logrotate 2024-03-23T07:59:07.538 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:59:07.546 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T07:59:07.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-23T07:59:07.776 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-23T07:59:07.872 INFO:tasks.ceph:Compressing logs... 2024-03-23T07:59:07.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T07:59:07.880 DEBUG:teuthology.orchestra.run.smithi132:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T07:59:11.685 INFO:tasks.ceph:Archiving logs... 2024-03-23T07:59:11.685 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/remote/smithi105/log 2024-03-23T07:59:11.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T07:59:12.372 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/remote/smithi132/log 2024-03-23T07:59:12.373 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T07:59:12.856 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T07:59:12.866 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T07:59:12.866 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T07:59:12.874 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T07:59:13.026 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T07:59:13.035 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T07:59:13.035 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T07:59:13.043 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T07:59:13.088 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-23T07:59:13.096 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T07:59:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-23T07:59:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 -282us[ -282us] +/- 74ms 2024-03-23T07:59:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -45us[ -42us] +/- 52ms 2024-03-23T07:59:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +426us[ +430us] +/- 91ms 2024-03-23T07:59:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:59:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-23T07:59:13.107 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T07:59:13.107 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-23T07:59:13.107 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 -257us[ -242us] +/- 74ms 2024-03-23T07:59:13.107 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -21us[-8627ns] +/- 52ms 2024-03-23T07:59:13.107 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +435us[ +448us] +/- 91ms 2024-03-23T07:59:13.107 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T07:59:13.109 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T07:59:13.118 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T07:59:13.118 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T07:59:13.127 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T07:59:13.341 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T07:59:13.556 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T07:59:13.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:59:13.660 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T07:59:13.759 INFO:teuthology.orchestra.run.smithi132.stdout:type=AVC msg=audit(1711179953.258:196): avc: denied { checkpoint_restore } for pid=1177 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-03-23T07:59:13.761 DEBUG:teuthology.task.selinux:ubuntu@smithi132.front.sepia.ceph.com has 1 denials 2024-03-23T07:59:13.761 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi132.front.sepia.ceph.com: ['type=AVC msg=audit(1711179953.258:196): avc: denied { checkpoint_restore } for pid=1177 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-23T07:59:13.762 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T07:59:13.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T07:59:13.782 INFO:teuthology.task.internal:Duration was 801.559760 seconds 2024-03-23T07:59:13.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T07:59:13.792 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T07:59:13.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T07:59:13.799 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T07:59:13.956 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T07:59:13.964 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T07:59:14.468 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T07:59:14.469 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-03-23T07:59:14.469 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T07:59:14.532 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-03-23T07:59:14.532 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T07:59:14.607 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T07:59:14.607 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T07:59:14.615 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T07:59:14.707 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T07:59:14.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T07:59:14.713 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T07:59:14.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T07:59:14.903 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T07:59:14.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T07:59:14.995 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T07:59:15.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T07:59:15.103 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T07:59:15.110 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T07:59:15.212 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-03-23T07:59:15.220 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-03-23T07:59:15.272 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T07:59:15.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:59:15.321 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T07:59:15.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:59:15.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T07:59:15.385 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T07:59:15.386 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/remote/smithi105 2024-03-23T07:59:15.386 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T07:59:15.491 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362/remote/smithi132 2024-03-23T07:59:15.491 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T07:59:15.592 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T07:59:15.592 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T07:59:15.600 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T07:59:15.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T07:59:15.664 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T07:59:15.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T07:59:15.674 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T07:59:15.675 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T07:59:15.683 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T07:59:15.730 INFO:teuthology.orchestra.run.smithi105.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 07:59 /home/ubuntu/cephtest 2024-03-23T07:59:15.738 INFO:teuthology.orchestra.run.smithi132.stdout: 393833 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 07:59 /home/ubuntu/cephtest 2024-03-23T07:59:15.741 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T07:59:15.752 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T07:59:15.763 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi132.front.sepia.ceph.com: ['type=AVC msg=audit(1711179953.258:196): avc: denied { checkpoint_restore } for pid=1177 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-23T07:59:15.763 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T07:59:15.796 DEBUG:teuthology.nuke:shortname: smithi105 2024-03-23T07:59:15.796 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T07:59:15.817 DEBUG:teuthology.nuke:shortname: smithi132 2024-03-23T07:59:15.817 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T07:59:15.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:36:50.027132', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCvGzdXZnCG7efrJf2Bl6Dzu60db9fLBm1CeDGCfKRjVrXXEK7w2OVUopAP2ZAodOEUU7d+cxO2ZO98vccUPrtE='} 2024-03-23T07:59:15.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:36:50.029059', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjJfB8Bx/I2rR6IbWMR3RU/LMDkgYFJUwA6VV18wAW1BPYbLlpVCjOp/Htuc5OR++0aJno1/ErdH334QNCUea0='} 2024-03-23T07:59:15.870 INFO:teuthology.orchestra.console.smithi105:Power off 2024-03-23T07:59:15.871 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T07:59:15.892 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-03-23T07:59:15.893 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:15.908 INFO:teuthology.orchestra.console.smithi132:Power off 2024-03-23T07:59:15.908 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T07:59:15.917 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-23T07:59:15.930 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2024-03-23T07:59:15.930 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:15.955 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-23T07:59:19.918 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:19.957 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:20.030 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-23T07:59:20.068 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-23T07:59:24.031 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:24.068 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:24.143 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-23T07:59:24.179 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-23T07:59:28.144 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:28.181 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T07:59:28.254 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-03-23T07:59:28.291 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2024-03-23T07:59:28.355 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-03-23T07:59:28.392 INFO:teuthology.orchestra.console.smithi132:Power off completed 2024-03-23T07:59:28.497 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 client.1-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/cap-flush} duration: 801.5597598552704 failure_reason: 'SELinux denials found on ubuntu@smithi132.front.sepia.ceph.com: [''type=AVC msg=audit(1711179953.258:196): avc: denied { checkpoint_restore } for pid=1177 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'']' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-03-23T07:59:28.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T07:59:28.586 INFO:teuthology.run:FAIL